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Applied  linguistics  and  academic  literacy 


It  is  with  great  pleasure  that,  as  the  guest  editor  of  this  special  edition  of  Ensovoort, 
I write  this  introductory  note.  I wish  to  record  at  the  outset  not  only  my  own 
appreciation,  but  also  that  of  my  department,  the  Unit  for  Academic  Literacy 
(UAL),  as  well  as  that  of  the  various  contributors,  from  several  different  institu- 
tions  of  higher  education,  to  the  editor-in-chief,  Johann  Lodewyk  Marais,  who 
generously  afforded  us  this  opportunity. 

We  are  privileged  to  be  able  to  bring  these  contributions  together  in  a volume 
dedicated  to  issues  of  literacy  generally,  and  to  that  of  academic  literacy  in  partic- 
ular.  All  of  the  contributionp  were  handpicked  either  from  presentations  that 
were  heard  by  members  of  the  UAL,  or  were  invited  from  researchers  that  we 
knew  were  doing  new  and  fresh  work  within  the  broader  field  of  applied  lin- 
guistics. 

The  responsible  work  that  is  being  done  within  the  sub-field  of  academic 
literacy  almost  without  fail  is  related  to  theoreticaJ  frameworks  and  pursuits  within 
the  larger  field  of  applied  linguistics.  Academic  literacy  itself  is  no  longer  the 
domain  of  those  who  only  wish  to  do  good  or  to  make  some  overt  political  state- 
ment.  Rather,  it  is  a growing  field  involving  those  who  intend  to  go  about  their 
business  of  facilitating  the  language  development  of  students  in  as  theoretically 
and  socially  responsible  a way  as  possible.  As  a recent  external  evaluation  of  our 
department  has  indicated,  teaching  on  its  own  will  never  be  sufficient  to  give  an 
adequate  response  to  the  needs  of  the  students  at  the  receiving  end  of  our  lan- 
guage  instruction:  our  teaching  needs  to  be  informed  by  research  of  the  highest 
quality  in  order  not  to  stagnate  or  deteriorate  into  smugness. 

I trust  that  the  contributions  in  this  special  volume  will  meet  with  these  and 
with  the  reader's  expectations  in  this  regard.  While  three  of  the  contributions  on 
academic  literacy,  those  of  Butler,  of  Louw,  and  of  Van  Dyk  et  al.,  deal  with  the 
designed  solutions  to  low  levels  of  academic  literacy  in  writing  instruction  and 
e-learning,  it  is  telling  that  the  four  others,  by,  respectively,  Cliff,  Ramaboa  and 
Pearce,  Geldenhuys,  Van  der  Slik  and  Weideman,  and  Van  der  Walt  and  Steyn, 
address  the  initial  diagnosis  of  the  problem:  how  these  levels  of  literacy  are  meas- 
ured  in  the  first  place.  In  both  sets  of  contributions,  the  increasing  sophistication 
with  which  academic  literacy  levels  is  measured,  and  the  research-backed  solu- 
tions  that  are  designed  to  address  inadequate  competence,  is  very  much  in  evi- 
dence. 
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The  contribution  of  Coetzee-van  Rooy  stays  within  the  domain  of  language  in 
higher  education,  but  examines  its  use  for  administrative  purposes,  while  the 
remaining  two  come  at  the  problem  obliquely:  Carstens  deals  with  the  com- 
prehensibility  of  health  education  materials,  and  Pretorius  with  early  literacy  in 
an  environment  characterised  by  poverty. 

We  are  grateful  to  the  referees,  who  have  done  an  excellent  job,  and  have, 
through  their  comments,  significantly  enhanced  the  quality  of  the  various  con- 
tributions. 

Jurie  Geldenhuys 

Pretoria,  6 December  2007 


To  flout  or  not  to  flout  - academic  writing  conventions  in 
tertiary  education 

H.G.  Butler 
University  of  Pretoria 


To  flout  or  not  to  flout  - academic  writing  conventions  in  tertiary  education 

As  a result  ofits  pnmary  importance  as  a vehiclefor  demonstrating students' 
academic  competence  in  tertiary  education,  the  ability  to  write  acceptable  academ- 
ic  texts  has  been  a topicfor  discussion  in  the  development  of  academic  literacyfor 
a number  ofyears.  In  recent  times,  such  discussion  has  awarded  prominence  to 
students' seeming mability  to  make  effedive  use  of  academic  discourse  in  both 
receptive  and  productive  modes  in  their  studies.  Whereas  one  may  expect  students 
new  to  the  tertiary  environment  to  be  relatively  inexperienced  academic  writers, 
one generally  expeds  postgraduate  students  with  their  considerable  experience  of 
university  study  to  be  proficient  academic  writers.  This,  however,  appears  to  be  an 
erroneous  assumption.  What  complicates  the  matter further  is  that  academic 
discourse  used  by  academic  writers  can  by  no  means  be  considered  to  be  a homoge- 
nous  phenomenon  in  the  tertiary  academic  context.  Although  one  would  expect 
writing  norms  and  conventions  to  be  relatively  similar  across  disciplines, 
academics  tend  to  adapt  the  discourse  used  in  their  specific  disciplines  to  suit  the 
needs  ofthe  discipline.  Itfollows  logically  that  efforts  to  support  students  with 
their  academic  writing  should  be  cognizant  ofwhat  is  required  ofstudents 
(preferably  in  their  specific  disciplines).  As  an  attempt  at  conceptualising  relevant 
writing  support,  this  paper  explores  the  notion  that  academic  discourse  is  governed 
by  normative  conditions  thatfind  expression  in  the  conventions  and  characteris- 
tics  ofacademic  texts.  It  further  presents  a critical  discussion  ofsome  ofthose 
characteristic  features  that  have  been  mooted  as  generic  to  academic  discourse.  In 
conclusion,  it  reflects  on  extreme  versions  ofcritical  literacy  theory  that  encourage 
students  to  challenge  the  conventions  and  dominant  norms  of  academic  discourse 
in  their  writing. 


1.  Introduction 

This  article  has  its  origin  in  a recently  completed  doctoral  study  on  the  design  of 
solutions  for  a pertinent  applied  linguistics  problem  - the  offering  of  appropriate 
academic  writing  support  to  tertiary  students.  Based  on  the  copious  amounts  of 
literature  available  on  the  topic,  it  is  clear  that  this  problem  is  not  restricted  to  the 
South  African  tertiary  context,  but  is  a difficulty  that  students  appear  to  experi- 
ence  at  universities  internationally.  The  topic  for  the  article  comes  from  a context 
where,  being  new  to  the  field  of  academic  literacy  (and  writing)  support,  I had  to 
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conceptualise  a writing  intervention  for  postgraduate  students  at  the  University 
of  Pretoria  (UP).  One  of  the  more  obvious  issues  at  the  time  was  to  decide  about 
the  content  and  nature  of  such  a course,  in  other  words,  what  it  was  that  one 
wished  to  address  and  how  such  an  intervention  had  to  be  designed  to  provide 
the  best  possible  leaming  opportunity  for  students.  As  an  initial  step  I set  out  to 
investigate  whether  any  generalisable  features/characteristics  could  be  said  to 
form  part  of  academic  discourse  in  order  to  decide  about  important  aspects  to 
include  in  a generic  postgraduate  writing  course.  This  paper  is  an  account  of  the 
questions  I felt  compelled  to  ask  about  some  of  the  more  prominent,  traditional 
features  of  written  academic  discourse  contained  in  various  guide  books  and 
courses  on  the  topic.  It  is,  however,  important  to  mention  at  the  outset  that  it  is 
difficult  to  generalize  about  what  characterizes  academic  texts,  because  of  the 
diverse  application  of  these  features  in  different  disciplines. 

Being  a long  time  convert  to  functional  linguistics,  it  was  important  for  me  to 
assess  prominent  textual  features  for  their  functionality  in  the  context  of  writing 
'academically'.  I therefore  wanted  to  determine  whether  such  features  had  real 
functional  value  in  terms  of  what  was  accomplished  through  their  use,  or  wheth- 
er  they  primarily  formed  part  of  a long  tradition  perpetuated  by  those  in  power  at 
specific  times  in  specific  discourse  communities.  What  mainly  prompted  this 
enquiry  was  some  of  the  earlier  work  in  critical  literacy  (cf.  Bizzel,  1992)  that 
questioned  academic  writing  conventions  on  the  basis  that  they  were  used  as  a 
device  to  maintain  the  status  quo  with  regard  to  power  relations  and  to  exclude 
minority  groups  from  participation  in  the  academic  discourse  community.  It  was 
therefore  important  for  me  to  determine  whether  such  criticism  was  primarily 
part  of  a political  agenda  of  select  individuals  or  if  academic  writing  conventions 
were  without  substantial  function  inherently,  and  thus  merely  part  of  the  ongo- 
ing  tradition  of  writing  in  a tertiary  context.  I do  believe  that  many  academics 
(usually  in  disciplines  other  than  language)  understandably  perpetuate  charac- 
teristics/features  of  written  academic  texts  in  their  disciplines  without  really  ques- 
tioning  their  applicability  and  value  for  their  current  context.  My  greatest  con- 
cem  is  whether  we  are  not  merely  subjecting  students  to  learn  to  write  'as  we  do' 
because  of  the  writing  tradition  we  were  subjected  to,  reverting  to  a largely  irrel- 
evant  initiation  ritual  into  the  'world  of  the  academy'.  Having  said  this,  I want  to 
emphasise  that  I do  not  wish  to  pass  judgment  on  the  value  of  tradition,  but 
rather  suggest  that  one  should  be  heedful  of  tradition  and  its  continued  utility  for 
changing  circumstances  over  time. 
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2.  Textual  conventions  of  academic  discourse 

From  the  preceding  discussion  it  is  apparent  that  if  one  wants  to  refer  to  academic 
discourse  as  an  objective,  factual  entity  on  its  own,  one  should  be  able  to  say  what 
it  is  and,  therefore,  which  distinctive  features  (should)  characterise  such  dis- 
course.  Determining  the  academic  writing  conventions  on  a textual  (lexical,  gram- 
matical,  stylistic)  level  for  the  whole  of  the  academic  discourse  community  would, 
however,  be  a mammoth  task  to  accomplish.  We  do  of  course  have  access  to  texts 
produced  all  over  the  world  in  an  academic  context  and  should  be  able  to  infer 
certain  generic  features  from  such  texts  (this  is  part  of  the  focus  of  some  of  the 
more  recent  studies  in  the  field  of  corpus  linguistics  [cf.  Conrad,  1996;  Biber, 
Johansson,  Leech,  Conrad,  & Finegan,  1999]).  The  descriptions  of  the  textual  char- 
acteristics  featuring  in  academic  writing  courses  and  manuals  may  be  determined, 
therefore,  by  conducting  text  investigations.  In  addition,  in  determining  the  typ- 
ical  identity  of  academic  discourse,  one  should  ask  not  only  whether  such  fea- 
tures  are  limited  to  academic  discourse,  but  also  whether  they  form  part  of  other 
types  of  discourse  (cf.  Hyland,  2000).  It  is  obvious  that  in  the  case  of  a requirement 
such  as  the  formality  of  academic  writing,  for  example,  academic  discourse  shares 
this  characteristic  with  much  occupational  writing  (technical  report  writing  in 
engineering,  for  example,  as  well  as  much  of  business  communication).  A more 
realistic  deduction  regarding  academic  writing  as  a separate  discourse  would  be 
that  on  a textual  level,  features  also  found  in  other  discourses  are  combined  in 
such  a way  as  to  form  what  could  be  referred  to  as  academic  discourse. 

Sometimes  the  suitability  of  certain  surface  structure  academic  writing  con- 
ventions  is  questionable.  At  times,  there  appears  to  be  a mismatch  between  what 
academic  language  is  supposed  to  accomplish  and  the  actual  language  forms  that 
are  used  in  written  academic  English,  for  example.  In  addition,  some  of  the  tradi- 
tional  features  of  academic  texts  at  times  seem  to  be  conflicting  in  terms  of  what 
writers  supposedly  wish  to  accomplish  by  employing  such  features.  The  follow- 
ing  discussion  focuses  on  some  of  the  traditional  features  of  academic  discourse. 
Although  criticism  is  offered  regarding  the  value/functionality  of  some  of  these 
characteristics,  it  does  not  necessarily  imply  that  students  should  be  encouraged 
to  flout  them  deliberately  (as  is  suggested  by  extreme  versions  of  a critical  literacy 
approach,  cf.  Bizzell,  1992),  especially  in  cases  where  they  are  deeply  entrenched 
in  some  disciplinary  discourse.  The  aim  of  the  following  discussion,  therefore,  is 
to  assess  the  value  of  such  features  in  purposefully  contributing  to  the  'academic' 
quality  of  written  texts,  in  other  words,  whether  they  are  used  in  response  to 
some  normative  condition  with  specific  functionality  in  the  tertiary  academic 
context. 
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2.1  Formality 

One  of  the  rnore  prominent  stylistic  features  found  in  various  guides  and  work- 
books  on  acadernic  writing  is  the  notion  that  it  makes  use  of  a formal  register.  As 
a number  of  authors  note  (amongst  others  Swales  & Feak  (1994),  Henning,  Gravett 
& van  Rensburg  (2002)  and  Coffin  et  al.  (2003)),  this  can  be  seen  most  visibly  in  the 
choice  of  lexical  items  used  in  this  type  of  discourse,  where,  for  example,  if  there 
is  a choice  between  a more  informal  and  a more  formal  word,  the  default  choice 
would  usually  be  the  formal  option.  Swales  and  Feak  (1994)  refer  to  this  feature  as 
a 'vocabulary  shift'.  In  this  regard,  the  use  of  words  that  are  characterised  as 
colloquialisms  and  slang  language  are  generally  not  appropriate  in  academic 
writing.  The  important  question  regarding  such  formality  is  whether  there  is  a 
substantial  functional  difference  should  students  be  allowed  to  use  a more  infor- 
mal  register  in  their  academic  texts.  After  all,  apart  from  the  fact  that  one  may 
initially  misunderstand  student  texts  in  the  case  of  informal  writing  (which  is 
rather  doubtful  since  most  academics  are  usually  experienced  academic  readers), 
one  should  ask  what  functional  purpose  it  serves  for  academic  writing  to  be 
formal.  Is  the  creation  of  a sense  of  seriousness  and  that  academics  are  engaged  in 
what  may  be  perceived  to  be  important  matters  really  that  important  functional- 
ly?  It  is  probably  this  sense  of  seriousness,  the  awareness  that  one  is  dealing, 
through  language,  with  issues  that  are  generally  true,  that  Blanton  (1998)  is  char- 
acterising,  amongst  other  things,  when  she  speaks  about  academic  discourse  hav- 
ing'authority'.  'Fhe  formality  that  is  so  often  mooted  as  a characteristic  of  academ- 
ic  discourse  no  doubt  serves  to  enhance  the  authoritativeness  of  the  claims  made 
in  such  language.  The  main  point  is  that  formality  per  se  is  not  a characteristic  of 
acadernic  discourse,  but  becornes  such  a feature  when  it  is  used  for  an  academic 
purpose  and  with  academic  intent,  viz.  the  condition  of  appropriateness  and 
authoritativeness.  The  functionality  gives  a typical  academic  purpose  to  the  for- 
mality. 

2.2  Conciseness  and  exactness 

A second  feature  of  acadernic  texts  is  that  they  are  supposed  to  be  as  to  the  point 
and  exact  as  possible.  Connected  to  the  feature  of  formality,  the  use  of  indetermi- 
nate/ vague  lexical  items  such  as  'thing'  and  'something'  is,  therefore,  not  usually 
exact  enough  to  be  acceptable  in  academic  writing.  Along  the  same  lines,  verbos- 
ity  and  redundancy  (such  as  unnecessary  repetition)  clutter  academic  argumen- 
tation  and  are  not  supposed  to  be  surface  features  of  academic  texts.  However,  it  is 
interesting  that,  for  example,  the  general  avoidance  of  first  person  pronouns  and 
contractions  so  often  mooted  as  being  stylistic  features  of  academic  writing  (cf. 
Biber  et  al,  1999)  contradicts  this  convention  because  such  structures  are  often 
replaced  by  longer  strings  of  words/letters,  rendering  the  text  less  economical. 
The  avoidance  of  the  latter,  however,  supports  the  feature  of  formality  referred  to 
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above,  since  they  are  usually  associated  with  more  casual,  informal,  less  severely 
constrained  forms  of  discourse,  such  as  conversations  among  equals.  Again,  in 
this  case,  the  condition  of  clear  argumentation  governs  the  use  of  language  in 
academic  texts. 

2.3  Impersorulity 

Coffin  et  al.  (2003:29)  remark  that:  "For  much  of  the  twentieth  century,  particular- 
ly  in  the  sciences,  the  notion  of  objectivity  meant  that  there  was  no  place  for  a 
personal  voice."  One  needs  to  question  whether  it  really  leads  to  more  objective 
writingif  one  refers  to  oneself  as  'the  author'  and  not  T,  for  example.  Therefore, 
does  it  lead  to  more  subjective  writing  if  the  author  personalises  a text?  Is  the 
quality  of  research  not  rather  to  be  found  in  how  the  research  was  structured  and 
conducted,  or  in  its  content?  In  fact,  the  more  recent  view  is  that  for  students  to 
become  competent  authors  of  academic  discourse,  they  need  to  achieve  their 
own  'voice',  i.e.  express  their  own  identity  (cf.  Ivanic  & Simpson,  1992).  Again,  an 
(emerging)  material  condition  finds  expression  in  the  formal  features  of  language, 
i.e.  when  we  actually  encourage  (newly  initiated)  academic  writers  to  use  the 
personal  pronoun.  And  without  'voice',  there  is  no  critical  thinking,  the  hallmark 
of  academic  reasoning.  It  is  important  to  note  though  that  there  are  instances 
where  students  'hide'  behind  their  personal  voices.  Reading  personalised  aca- 
demic  texts,  it  becomes  clear  that  such  students  are  rather  uninformed  about 
related  literature  - they  therefore  merely  state  what  they  know  about  an  issue.  It 
is  thus  important  that  students  be  made  aware  of  a balanced  argument  that  incor- 
porates  authoritative  views  on  issues  as  well  as  their  own. 

In  a related  issue,  the  use  of  the  passive  is  normally  supposed  to  make  writing 
more  impersonal  (an  important  traditional  feature  of  academic  writing),  yet  sourc- 
es  on  academic  writing  differ  about  whether  using  passives  is  a good  practice  in 
such  writing.  Academic  texts  are  written  at  and  for  different  levels  of  accessibility, 
and  we  may  therefore  in  some  cases  wish  to  avoid  passives  in  order  to  write  more 
intelligibly. 

2.4  Nominalisation 

Another  important  feature  of  academic  discourse  is  the  degree  of  nominalisation 
that  typically  characterises  such  texts.  Ventola  (1998:68)  maintains  that  scientific 
language  has  evolved  over  time  to  suit  the  needs  of  those  who  practice  it.  She 
explains  this  change  as  follows: 

The  grammar  of  scientific  language  has  changed  as  reporting  about  scientific 

experiments  and  processes  have  developed.  Thoughts  are  now  foregrounded. 

Dynamic  actions  have  become  static,  intellectualised,  when  grammatical  roles 

have  changed,  through  nominalization,  from  processes  or  events  into  things. 
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The  discourse  act  that  involves  the  nominalisation  of  processes,  of  course,  makes 
it  possible  for  academic  discourse  to  create  abstractions  - something  that  Martin 
and  Rose  (2003:103)  refer  to  as  'ideational  metaphor'.  They  explain  metaphor  in 
general  as  "a  transference  of  meaning  in  which  a lexical  item  that  normally  means 
one  thing  comes  to  mean  another"  (Martin  & Rose,  2003:103).  For  them,  ideational 
metaphor  involves  a transference  of  meaning  from  one  kind  of  element  (in  this 
case  a grammatical  element)  to  another.  The  example  they  provide  clearly  illus- 
trates  this  shift  in  meaning,  where  a process  such  as  marrying  can  also  be  treated 
as  a quality,  married,  as  well  as  a thing-  marriage.  These  authors  further  explain 
that  in  modern  written  languages,  the  shift  in  meaning  accomplished  when  us- 
ing  a strategy  such  as  nominalisation  expands  the  set  of  meanings  available  to 
writers.  In  essence,  the  creation  of  an  abstraction  that  is  achieved  through  nomi- 
nalisation  serves  the  purpose  that  is  central  to  theorising,  conceptualisation  and 
argumentation  in  academic  writing,  viz.  distinction-making. 

What  is  further  evident  is  that  a high  degree  of  nominalisation  is  one  of  the 
features  of  academic  writing  that,  because  it  makes  the  language  more  complex 
(through  a strategy  of  concision),  it  also  renders  it  less  readable  (and,  therefore, 
accessible),  espedally  to  those  who  do  not  yet  form  part  of  the  academic  discourse 
community.  Although  students  new  to  this  environment  might  have  had  some 
limited  exposure  to  information-dense  academic  texts,  this  is  one  of  the  obstacles 
that  denies  many  students,  especially  additional  language  users,  access  to  the 
tertiary  environment.  It  might  also  be  interesting  to  note  that  again,  an  important 
feature  of  academic  writing  - in  this  case  its  inf  ormation  density  - seems  to  neg- 
atively  affect  another  feature,  its  clarity,  with  regard  to  how  students  new  to  this 
environment  struggle  to  unlock  the  meaning  in  such  texts.  Relevant  support  to 
enable  students  to  unlock  such  texts  productively  seems  unavoidable  if  many 
new  students  are  to  succeed  with  their  studies  in  this  environment. 

2.5  Grammatical  correctness 

Grammatical  correctness  of  academic  texts  is  supposed  to  be  non-negotiable  in 
the  academic  world.  Student  writing,  however,  often  appears  to  be  riddled  with 
grammatical  errors.  The  question  should  be  asked  then  why  very  little  evidence 
exists  to  suggest  that  lecturers  from  disciplines  other  than  language  pay  any  at- 
tention  to  grammar  when  they  mark  undergraduate  student  scripts,  or  when 
they  do,  they  do  so  in  a highly  selective  way,  focusing  on  one  or  two  grammatical 
features  (e.g.  tense,  concord)  only.  These  lecturers  in  some  way  still  seem  to  un- 
derstand  student  writing,  which  indicates  that  the  communicative  requirement, 
viz.  conveying  the  appropriate  information,  in  this  case  from  student  to  lecturer, 
is  being  met.  This  issue  could  probably  also  be  connected  to  that  of  coherence  in 
student  writing  discussed  under  the  next  point.  This  feature  may  further  be  linked 
to  the  conditions  of  authority  and  clarity  of  meaning.  It  would  be  difficult  to  be 
taken  seriously  if  one's  written  text  is  riddled  with  errors. 
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2.6  Coherent  and  cohesive  argvmentation 

Coherence  and  cohesion  in  academic  writing  are  mostíy  created  by  the  purpose- 
ful  use  of  connecting  devices  that  highlight  the  flow  of  ideas  and  signal  the 
writer's  intentions  regarding  the  specific  relationships  between  such  ideas.  While 
cohesion  usually  involves  sentential  and  ideational  connection  within  the  text, 
coherence  refers  to  the  overall  organisation  of  text  into  a recognisable  sequence 
(e.g.  text  development  from  the  introduction  to  conclusion).  Prosser  and  Webb 
(1993)  refer  to  specific  devices  used  to  create  a predictable  text  structure  (such  as 
presenting  all  the  main  sections  of  an  essay  in  the  introduction,  thereby  creating 
an  expectation  as  to  what  will  follow)  as  'predictive  scaffolding'.  Proficient  aca- 
demic  writers  make  use  of  such  devices  in  order  to  lead  readers  through  a text, 
also  showing  awareness  of  the  fact  that  academic  readers  will  probably  know  the 
textual  pattems  of  academic  texts  and  therefore  find  it  easier  to  understand  texts 
organised  in  this  manner.  Formulated  in  ethnomethodological  terms:  competent 
academic  writers  (and  the  readers  of  their  texts)  have  an  orientation  to  something 
that  we  may  term  an  argumentative  schema  or  framework.  Once  this  framework 
is  activated,  e.g.  through  the  use  of  discourse  markers,  the  text  becomes  more 
intelligible.  We  again  have  an  instance  here  of  how  a factual  feature  of  academic 
texts,  in  this  case  coherence,  is  determined  by  a norm  or  condition  - the  orienta- 
tion  towards  an  argumentative  framework. 

Given  the  number  of  complaints  by  lecturers  about  students  producing  inco- 
herent  texts  (especially  at  postgraduate  level),  one  could  ask  whether  this  issue 
might  not  also  be  related  to  ways  in  which  lecturers  generally  read  and  respond 
to  student  texts.  Do  lecturers  read  student  scripts  for  fluent  argumentation,  or  are 
assessment  opportunities  arranged  in  such  a way  that  only  fragmented  chunks  of 
knowledge  are  often  required  of  leamers  and  therefore  acknowledged  by  lectur- 
ers?  If  so,  this  is  a clear  example  where  students'  overall  literacy  development  is 
neglected  by  lecturers  in  their  undergraduate  years  and  when  supervisors  re- 
quire  language  fluency  and  correctness  on  a postgraduate  level,  they  suffer  the 
consequences  of  such  neglect. 

2. 7 Appropriate  use  ofevidence 

Academic  writingshows  certain  conventions  with  regard  to  how  the  ideas/words 
of  authorities  (other  sources)  are  acknowledged.  Although  different  referencing 
systems  are  used  across  the  world,  what  is  shared  by  academic  writing  (in  a 
western  context)  is  that  other  people's  ideas  should  be  overtíy  acknowledged  in 
one's  own  academic  writing.  It  is  interesting  that  the  notion  of  writing  and  ideas 
as  the  individual's  'property'  is  not  always  shared  by  all  cultures,  espedally  where, 
historically,  the  development  of  ideas  and  knowledge  has  taken  a different  route. 
In  China,  for  example,  a learned  person  is  often  recognised  as  someone  who  can 
memorise  information  very  well,  espedally  regarding  texts  that  classical  authors 
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wrote.  As  a consequence,  such  texts  become  part  of  the  person's  memory  and  are 
supposed  to  be  recognised  by  other  leamed  people  without  it  being  necessary  for 
anyone  to  state  explicitly  that  the  words  were  initially  spoken  or  written  by 
somebody  else. 

Another  interesting  perspective  on  the  issue  of  plagiarism  is  that  of  Angelil- 
Carter  (2000)  who  notes  that  neophyte  writers  may  be  making  use  of  sources  as 
models  for  meeting  specific  written  conventions  and  norms  of  the  academic  dis- 
course  community.  Although  they  may  not  necessarily  want  to  copy  the  ideas  of 
a source,  they  may  want  to  copy  the  way  in  which  language  is  used  by  the  source. 
So,  while  violating  one  of  the  most  important  conditions  of  academic  writing  on 
the  one  hand,  they  might  be  striving  to  meet  another,  that  of  the  appropriate  use 
of  language,  on  the  other.  In  this  case,  a degree  of  flexibility  is  called  for  in  under- 
standing  the  predicament  of  writers  new  to  this  environment  and  that  this  kind 
of  copying  might  form  part  of  their  process  of  becoming  more  proficient  academ- 
ic  writers. 

What  lecturers  require,  however,  is  that  references  should  be  purposefully 
integrated  into  the  text  in  support  of  the  writer's  argument,  and  not  just  be  a 
collection  of  quotes  without  relationship  or  interpretation.  Again,  the  idea  of  aca- 
demic  writing  being  framed  by  the  notion  of  a structured  argument  is  evident: 
references  are  used  to  support  one's  argument.  Similarly,  the  concept  of  authority 
comes  to  the  fore:  in  order  to  enhance  the  authority  of  one's  own  academic  text, 
one  supports  it  with  reference  to  that  of  an  already  acknowledged  authority.  Ulti- 
mately,  the  rhetorical  purpose  of  arguing  with  authority  (in  specific  genres)  may 
be  the  most  important  condition  of  academic  writing.  Other  genres  may  have 
another  rhetorical  purpose,  e.g.  the  laboratory  report  that  provides  an  account  of 
a scientific  procedure,  providing  specific  information  in  a predetermined  format. 


3.  Conclusion 

I have  tried  to  emphasise  in  this  article  the  notion  that  similar  to  any  other  type  of 
discourse,  academic  discourse  cannot  be  divorced  from  its  social  context.  Aca- 
demic  discourse  is  further  not  a homogeneous  entity,  but  varies  considerably 
across  and  even  within  disciplines  in  the  tertiary  academic  environment.  This 
variability  is  a crucial  feature  of  academic  discourse  that  should  inform  the  de- 
sign  of  writing  courses  in  university  education.  Nonetheless,  certain  key  norma- 
tive  features  of  academic  discourse  can  be  identified,  and  one  can  identify,  also, 
various  typical  features  of  academic  texts  that  are  regulated  by  such  normative 
conditions,  and  that  are  in  complex  interaction  with  one  another. 

Thus,  the  true  worth  of  extremities  in  theorising  (such  as  extreme  versions  of 
critical  literacy)  is  that  they  urge  one  to  revisit  critically  some  relatively  estab- 
lished,  traditional  notions  on  what  constitute  the  features  of  academic  discourse 
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(with  specific  reference  to  academic  writing).  Therefore,  in  the  case  of  extreme 
versions  of  critical  literacy,  they  accomplish  exactly  what  they  set  out  to  do,  viz.  to 
entice  critical  enquiry.  It  is  important  then  that  extreme  versions  of  critical  litera- 
cy  not  be  discarded  immediately  as  'too  radical'  but  be  carefully  assessed  with 
regard  to  focusing  critical  enquiry. 

I strongly  believe  that  identifying  a core  of  traditional  (functional)  character- 
istics  of  academic  discourse  serves  as  a necessary  foundation  for  initiating  discus- 
sions  with  various  role  players  in  specific  disciplines  towards  relevant  and  re- 
sponsible  writing  course  design  for  students  in  such  disciplines.  Such  issues 
could  then  be  discussed  and  negotiated  with  such  persons  in  order  to  ascertain 
their  specific  needs  and  to  ensure  as  far  as  possible  that  these  issues  are  contextu- 
alised  within  such  disciplines. 
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Comprehension  of  pictures  in  educational  materials 
for  HIV/AiDS 

Adelia  Carstens 
University  of  Pretoria 


Comprehension  of  pictures  in  educational  materials  for  HIV/AIDS 

The  article  reports  on  a research  project  that  was  aimed  at  determining  differ- 
ences  in  picture  comprehension  between  literate  and  low-literate  audiences  in 
the  context  ofHIV  and  AIDS.  Structured  interviews  were  held  with  30  low- 
literate  and  25  literate  adult  speakers  from  African  languages.  The  responses 
were  coded,  and  analysed.  Although  metaphorical  pictures  proved  to  be  proble- 
maticfor  both  literates  and  low-literates  the  low-literate  group  was  more 
inclined  towards  literal  interpretation  ofpictorial  metaphors,  and  culturally 
encoded  meanings  seemed  to  obfuscate  the  meaning  of  certain  commonly  used 
mass  media  symbols.  Pictures  containing  symbolic-abstract  components  posed 
particular  problems  for  the  low-literates,  and  interpretation  problems  were 
compounded  by  poor  legibility,  complexity  and  an  unclear  figure-ground 
distinction. 


1.  The  problem:  HIV/AIDS,  literacy  and  health  communication 

Effective  communication  is  the  backbone  of  health  promotion  and  disease 
prevention.  People  need  to  understand  health  information  to  apply  it  to  their 
own  behaviour.  Davis,  Michael,  Couch,  Wills,  Miller  & Abdehou  (1990:  533) 
regard  comprehension  as  the  most  important  of  all  the  literacy  skills  used  in 
health  care.  These  authors  found  in  their  research  in  the  United  States  that  the 
average  reading  comprehension  of  public  clinic  patients  was  the  6th  grade  5* 
month,  whereas  most  tested  patient  education  materials  required  a reading 
level  of  the  llth  to  the  14th  grade.  Forty  percent  of  all  public  clinic  patients 
tested  were  reading  below  a 5th  grade  level  and  could  be  considered  'severely 
illiterate'  (cf.  also  Plimpton  and  Root,  1994:  86).  The  South  African  situation  is 
comparable.  Basic  instructional  materials  on  health  issues  (including  HIV  and 
AIDS)  have  a readability  level  of  just  below  60,  which  is  equivalent  to  Grade  9 
(Carstens  & Snyman,  2003),  while  more  than  70%  of  the  South  African  popu- 
lation  have  only  marginal  reading  skills:  30%  are  functionally  illiterate  and 
the  other  40%  have  limited  skills  (Carstens,  2004;  Project  literacy,  2004).  A com- 
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pounding  factor  is  that,  as  a rule,  30-50%  of  low-literate  patients  read  3-5  years 
below  their  educational  level  (cf.  Davis  et  al,  1990:  535,  537).  Moreover,  the 
grade  level  reported  by  low-literate  audiences  is  often  adjusted  upward  by  a 
few  levels,  presumably  to  save  face. 

One  may  ask  why  formal  education  is  so  important  in  the  realm  of  health. 
The  answer  lies  in  the  fact  that  years  of  schooling  completed  is  one  of  the 
most  important  socioeconomic  correlates  of  good  health  in  adult  populations 
(Grosse  & Auffrey,  1989:  281).  The  most  poorly  educated  adults,  those  with 
the  lowest  literacy  levels,  suffer  the  highest  rates  of  morbidity  and  mortality 
from  chronic  diseases  and  conditions  (Rudd,  Moeykens  & Colton,  1999; 
Plimpton  & Root,  1994;  NWGLH,  1989).  This  correlation  can  be  explained  as 
follows:  If  people  cannot  understand  the  health  care  information  available  to 
them,  they  are  unable  to  change  potentially  harmful  behaviours,  and  improve 
their  health. 

In  developing  countries  such  as  South  Africa,  where  almost  two  thirds  of 
the  population  cannot  read  basic  health  education  materials,  the  solution  is 
often  sought  in  visual  media  (Arbuckle,  2004).  In  health  campaigns  across  the 
world  pictures  are  used  where  the  written  word  fails  to  communicate  effec- 
tively  - usually  to  supplement,  extend  or  reinforce  oral  instructions  (cf.  Doak 
et  al.,  1996:  92;  Mayeaux  et  al.,  205;  207).  Moreover,  various  studies  report  on 
the  successes  of  using  pictures  in  health  education  in  developing  countries 
(cf.  Hoffmann,  2000;  Linney,  1995;  PATH,  2002;  Plimpton  & Root,  1994;  Toma- 
selli  & Tomaselli,  1982;  Zimmermann,  1981). 

However,  there  are  deficiencies  in  the  research  conducted  thus  far: 

• It  has  not  been  proven  beyond  doubt  that  people  who  cannot  read  well, 
will  be  able  to  comprehend  and  learn  from  visual  communication  (Doak 
et  al,  1996:  92). 

• Almost  all  the  studies  on  the  interpretation  of  pictures  in  health  educa- 
tion  that  have  been  undertaken,  lack  a purposeful  theoretical  orienta- 
tion  and  a sound  theoretical  basis  (Hoffmann,  2000: 136).  Many  sources 
on  problematic  pictures  in  development  contexts  can  at  most  be  regarded 
as  anecdotal  accounts  or  hybrid  lists  of  the  difficulties  observed  (in- 
cluding  a variety  of  semantic,  syntactic,  pragmatic,  cognitive,  cultural 
and  stylistic,  problems)  (cf.  Colle  & Glass,  1986). 

• No  comparative  studies  have  been  done  to  prove  that  there  is  a signifi- 
cant  difference  in  the  comprehension  of  certain  types  of  pictures  be- 
tween  low-literate  and  literate  audiences. 

The  research  reported  on  in  this  article  was  particularly  aimed  at  establishing 

• whether  purely  analogical  (representational)  pictures  are  interpreted 
without  difficulty  by  literate  as  well  as  low-literate  audiences; 
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• whether  low-literate  audiences  experience  problems  interpreting  sym- 
bolic-analogical  pictures,  whereas  literate  audiences  experience  fewer 
problems  or  no  problems  at  all; 

• whether  low-literate  audiences  experience  problems  interpreting  sym- 
bolic-abstract  pictures,  whereas  literate  audiences  experience  fewer 
problems  or  no  problems  at  all. 


2.  Categories  of  pictures  often  reported  as  'difficult'  for  low-literates 

The  problems  mentioned  in  the  diverse  literature  on  picture  comprehension 

by  low-literate  audiences  can  be  roughly  categorised  as 

• pictures  with  too  much  distracting  detail  in  the  background,  causing 
the  unskilled  viewer  to  miss  the  central  focus  of  the  visual,  or  to  focus 
on  the  wrong  detail  (Ausburn  & Ausburn,  1983:  113;  Doak  et  al.,  1996: 
93;  103;  Linney,  1995:  23;  NCI,  1994;  PATH,  2002:  2); 

• pictures  that  are  misunderstood  due  to  differences  in  the  knowledge 
systems  of  the  author/compiler/designer  and  the  audience  (Bradley,  1995: 
1;  Cornwall,  1992;  Doak  et  al,  1996: 99;  Tomaselli  & Tomaselli,  1984;  Tripp- 
Reimer  & Afifi,  1989:  613); 

• pictures  reflecting  western  pictorial  conventions  of  depth  perspective, 
including  linear  perspective,  occlusion,  relative  size,  etc.  (Arbuckle,  2004; 
Bradley,  1995:  74;  Linney,  1995:  23-24;  PATH,  2002:  2); 

• pictures  representing  unseen  entities,  e.g. 

— objects  that  are  too  small  to  be  observed  by  the  human  eye,  such 
as  the  HI  virus; 

— objects  that  are  concealed  inside  an  outer  'shell',  such  as  internal 
organs; 

— the  operation  of  systems  (Colle  & Glass,  1986: 161;  Dudley  & Haa- 
land,  1993:  37); 

— depiction  of  movement,  such  as  a physical  object  traveling  from 
one  point  to  another,  emission  of  light,  steam,  breath,  sound,  etc., 
by  making  use  of  lines  (Arbuckle,  2004;  Colle  & Glass,  1986: 161; 
Hoffmann,  2000: 142;  PATH,  2002:  2) . 

• pictures  rendered  in  unfamiliar  art  styles,  e.g.  cartoon-style  drawings 
of  people  and  objects,  which  both  simplify  and  distort  (Doak  et  al,  1996: 
95;  Plimpton  & Root,  1994:  86); 

• complex  pictures  of  which  the  meaning  is  dependent  on  a particular 
sequence,  e.g.  left  to  right,  or  correct  interpretation  of  a particular  rela- 
tionship,  e.g.  causal,  temporal,  etc.  (Colle  & Glass,  1986: 161;  Haaland, 
1984;  Hoffmann,  2000:  95;  134;  142;  Linney,  1995:  23-25); 
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• pictures  containing  sjnnbols  from  popular  media  or  non-familiar  knowl- 
edge  domains,  e.g.: 

— metaphoric  symbols,  such  as  a heart  (love),  a dove  (peace),  etc. 
(Linney,  1995:  24;  PATH,  2002:  2).  (Although  they  are  representa- 
tional,  a transferred  meaningis  conveyed,  which  is  often  culture- 
dependent); 

— abstract  symbols  of  which  the  meanings  are  not  transparent,  but 
have  been  fixed  by  convention,  e.g.  road  signs,  arrows,  ticks,  cross- 
es,  mathematical  symbols,  circles  and  splashes  of  colour  (to  point 
out  or  indicate)  (Colle  & Glass,  1986:  161;  Doak  et  al.,  1996:  103; 
106;  Kress  & Van  Leeuwen,  1996:  61-70;  PATH,  2002: 2);  as  well  as 
certain  pictorial  conventions  used  in  the  cartoon  style,  e.g.  thought 
balloons  and  speech  balloons  (Colle  & Glass,  1986: 16;  Hoffmann, 
2000: 142;  PATH  2002:  2;  Hugo  & Skibbe,  1991:49). 

In  semiotics  signs  are  often  classified  as  either  symbolic  (where  the  relation- 
ship  between  the  signifier  and  the  signified  is  arbitrary)  or  analogical  (where 
the  relationship  between  the  signifier  and  the  signified  is  logical,  natural  or 
direct).  In  the  case  of  pictures,  'analogical'  normally  also  entails  literal,  and 
irrespective  of  the  literacy  level  of  the  viewer,  this  type  of  picture  is  usually 
only  misunderstood  if  the  viewer  is  not  familiar  with  the  object  portrayed. 

Following  Gralki  (1985),  Hoffmann  (2000: 8)  distinguishes  two  types  of  pic- 
tures  with  a partially  or  completely  symbolic  (arbitrary)  content,  namely  sym- 
bolic-analogical  and  symbolic-abstract.  These  types  of  pictures  are  not  direct 
representations  of  people's  reaLworld  perceptions.  In  other  words,  they  are 
not  purely  analogical.  Some  kind  of  cognitive  transformation,  based  on  aca- 
demic  or  cultural  knowledge,  is  needed  to  connect  them  to  their  intended 
meanings. 

Symbolic-abstract  representations  are  images  that  are  purely  fixed  by  con- 
vention.  Hoffmann  (2000:  85)  characterizes  them  by  saying  that  with  these 
images  "there  is  a constant  tendency  to  cross  the  line  into  the  field  of  written 
representation".  Figures,  formulae,  tables,  mathematical  symbols  (e.g.  the  con- 
ventionalized  symbols  for  equation,  addition,  subtraction,  multiplication,  di- 
vision,  etc.),  logical  notation  (e.g.  an  arrow  to  signify  entailment),  etc.,  are  ex- 
amples  of  signs  that  are  closer  to  written  signs  than  pictorial  signs.  Although 
not  mentioned  by  Hoffmann,  speech  balloons  and  thought  balloons  may  be 
categorised  as  symbolic-abstract  as  there  is  no  direct,  natural  or  logical  resem- 
blance  between  form  and  meaning  (apart  from  the  fact  that  they  are  'contain- 
ers'  of  embedded  meaning),  and  since  their  meanings  are  only  fixed  by  con- 
vention.  Despite  the  fact  that  signets  and  logos  may  have  originally  been  ana- 
logical  Hoffmann  categorizes  them  as  symbolic-abstract  once  they  have  be- 
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come  the  standard  symbols  for  companies  (e.g.  the  Mercedes  star),  products 
(the  international  wool  mark)  or  organizations  (e.g.  the  five  Olympic  rings, 
the  International  Red  Cross). 

Symbolic-analogical  pictures  constitute  a hybrid  category  between  sym- 
bolic  and  analogical.  These  signs  are  interpreted  symbolically  in  that  that  their 
use  is  fixed  by  convention,  and  analogically  in  that  some  structural  or  func- 
tional  resemblance  to  the  object  represented  is  preserved  in  the  meaning.  This 
category  includes 

• various  types  of  diagrams  which  people  use  to  convey  abstract  con- 
cepts,  such  as  quantities,  relationships  or  processes:  a bar  chart  shows  a 
comparison  between  separate  entities  of  different  sizes  (containers);  a 
line  graph  shows  relationships  between  elements  (evolution); 

• pictorial  metaphors,  such  as  a (schematic)  representation  of  the  heart  to 
represent  love;  a representation  of  a spider's  web  to  indicate  a URL/ 
Internet  site; 

• pictorial  metonymies,  in  the  case  of  which  the  picture  represents  an 
element  or  an  attribute  of  an  object,  action,  event  or  condition,  e.g.  a 
representation  of  a clock  to  represent  time  in  general;  a smiley  face  to 
represent  happiness  or  friendship;  a knife  and  fork  to  represent  a res- 
taurant,  a bed  to  represent  overnight  accornmodation  (cf.  Hoffmann, 
2000:  85). 

• Graphemes  with  fixed,  conventional  meanings  that  present  unseen 
entities,  e.g.  lines  to  indicate  movement,  sound,  light  or  heat,  etc.,  are 
analogical  in  a certain  way.  Although  these  lines  depict  imaginary  ob- 
jects  or  events,  scientific  knowledge  (e.g.  about  light  and  sound  waves, 
etc.)  partially  motivates  them. 

Although  the  exposition  given  above  suggests  that  the  categories  analogical/ 
literal,  symbolic-analogical  and  symbolic-abstract  are  non-discrete,  this  art- 
icle  will  treat  all  non-literal  pictures  as  either  belonging  to  the  category  sym- 
bolic-analogical  or  symbolic-abstract. 


3.  Research  methodology 

The  method  of  data-collection  is  primarily  qualitative  (structured  interviews), 
but  demonstrates  elements  of  an  experiment  since  the  same  interview  sched- 
ule  was  used  for  two  different  respondent  groups,  and  after  coding  the  data, 
the  two  groups  were  compared,  using  statistical  techniques. 
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3. 1 Respondents 

The  experimental  group  comprised  30  low-literate  speakers  (persons  with  lit- 
eracy  levels  below  9 years  of  formal  schooling)  of  8 African  languages  between 
the  ages  of  22  and  55  years,  and  the  control  group  comprised  25  literate  (with 
literacy  levels  above  9 years  of  formal  schooling)  speakers  of  8 African  lan- 
guages  aged  between  23  and  49. 

The  literacy  levels  of  the  respondents  were  determined  on  the  basis  of  self- 
reports  regarding  years  of  formal  schooling.  Eight  years  of  schooling  was  set 
as  the  upper  limit  for  the  experimental  group,  because  it  is  regulated  by  South 
Airican  law  that  learners  who  have  passed  Grade  9 may  leave  school  and  start 
tertiary  training.  A supporting  motivation  was  that  persons  with  fewer  than  9 
years  of  schooling  are  regarded  (in  terms  of  the  categories  defined  by  Project 
Literacy  2004)  as  only  'marginally  literate'.  The  lower  limit  for  the  control  group 
was  9 years  of  schooling  and  the  upper  limit  was  students  who  have  complet- 
ed  a first  degree. 

The  sampling  method  was  both  convenient  and  purposive,  as  the  data 
gatherers  relied  on  personal  acquaintances  to  identify  and  recruit  respond- 
ents  who  satisfied  the  literacy  requirements.  All  the  respondents  were  inter- 
viewed  individually.  Low-literate  respondents  were  interviewed  in  Constan- 
tia  Park  and  Waterkloof  Glen  in  Pretoria  as  well  as  in  KwaMhlanga,  and  liter- 
ate  respondents  were  interviewed  in  Pretoria  Central,  Menlo  Park,  Lynn- 
wood  and  Hatfield.1 

The  gender  imbalance  (25  females  vs.  5 males  in  the  experimental 
group;  and  18  females  vs.  7 males  in  the  control  group)  should  be  attrib- 
uted  to  the  sampling  strategy  (convenience). 

The  following  matrices  summarize  the  socio-demographic  profile 
of  the  respondents: 

I:  Socio-demographic  profile  of  the  experimental  group 

Years  of  formal  Gender  Occupation  Home  language 

education 


Female  25 

Domestic  workers  16 

IsiZulu  9 

Male  5 

Unemployed  7 

IsiNdebele  7 

Gardeners  4 

Sepedi  7 

Cleaners  3 

Sesotho  3 

Setswana  1 

Siswati  1 

Xitsonga  1 

IsiXhosa  1 

The  literate  respondents  were 
interviewed,  and  the  data  for 
the  literate  group  was  coded 
by  Ms.  L.  Birir-Gangla. 


Table  • 
Age 
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Table  2:  Socio-demographic  profile  of  the  control  group 

Age  Years  of  formal  Gender  Occupation  Home  language 

education 

Mean  32.8  Mean  12.6  Female  18  Domestic  workers  5 Isizulu  4 

Male  7 Administrative  workers  13  IsiNdebele  1 

Undergraduate  students  7 Sesotho  1 
Sepedi  1 
Setswana  1 
Siswati  1 
Tshivenda  1 
Other  15 

3.2  Materials 

The  materials  comprised  a compilation  of  fourteen  pictures  from  various  pub- 
lic  information  docurnents  on  f irV/AIDS  that  had  been  collected  from  educa- 
tional  and  public  health  care  facilities  (clinics,  hospitals,  schools)  in  and  around 
Pretoria  during  the  period  2000-2004.  The  pictures  were  scanned  and  arranged 
in  a narrative  sequence,  which  could  be  characterized  as  'the  story  of  AIDS'.  A 
narrative  sequence  was  chosen  to  introduce  logic  into  the  design,  in  the  ab- 
sence  of  written  text  or  oral  narration.  In  the  case  of  the  low-literate  respond- 
ents  a pre-interview  briefing  was  done  to  ensure  that  they  were  familiar  with 
the  main  concepts  regarding  HTV  and  AIDS. 

The  picture  sequence  included  pictures  on  talking  about  sex  and  pregnan- 
cy;  talking  about  sex  and  protection  against  HIV/AIDS;  postponing  sexual 
debut;  HIV/AIDS  and  pregnancy;  negotiating  condom  use;  HlV-testing  (and 
counselling);  regular  exercise;  getting  rest;  healthy  and  unhealthy  food  choic- 
es  (lunch);  prohibition  of  smoking  and  alcohol  use;  and  taking  antiretroviral 
medicines  according  to  schedule  (compare  Addendum  A). 

Due  to  the  availability  of  pictures  on  the  different  aspects  of  the  AIDS  nar- 
rative,  and  in  order  to  test  particular  picture  types,  the  portfolio  included  pic- 
tures  rendered  in  different  art  styles:  black  and  white  semi-realistic  line  draw- 
ings,  coloured,  cartoon-style  line-drawings;  coloured,  realistic  line  drawings 
with  background  detail;  a silhouette;  coloured,  semi-realistic  line  drawings 
without  background  detail;  shaded  colour  drawings  without  background  de- 
tail;  shaded  colour  drawings  with  background  detail. 

3.3  Data  collection 

The  data  were  collected  via  structured  interviews.  The  general  procedure  was 
to  start  each  interview  by  introducing  the  interviewer,  and  asking  about  the 
preferred  language  for  the  interview.  Interviewees  were  informed  that  their 
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responses  would  be  tape-recorded  anonymously,  that  their  participation  was 
voluntary,  and  that  they  were  entitled  to  withdraw  their  participation  at  any 
stage  during  the  research  process.  Each  respondent  was  asked  verbally  for 
his/her  consent  to  use  the  data,  and  to  proceed  with  the  interview. 

After  the  introduction,  the  respondents  were  briefed  on  the  purpose  of  the 
research,  namely  to  assist  the  researcher  in  finding  out  which  of  the  pictures 
should  be  included  in  health  education  materials  distributed  at  clinics. 

It  was  assumed  that  low  literate  participants  would  have  less  general  know- 
ledge  about  preventing  and  living  with  AIDS  than  literate  participants.  There- 
fore,  in  the  low-literate  condition  the  structured  interviews  were  preceded  by 
a semi-structured  briefing  (without  visual  support)  on  the  topic  of  HIV/AIDS, 
according  to  a schedule  covering  sexual  debut,  prevention  of  HIV/AIDS,  and 
coping  with  the  illness.  The  researcher  asked  questions,  confirmed  correct 
answers,  and  provided  correct  information  where  the  respondent  did  not  know 
the  answer  or  held  erroneous  beliefs,  in  order  to  create  sufficient  contextual 
knowledge  for  the  interpretation  of  the  pictures.  Respondents  were  invited  to 
ask  questions,  and  to  comment  on  any  of  the  issues  raised.  In  the  literate  con- 
dition,  this  phase  was  skipped. 

Subsequently,  socio-demographic  information  was  recorded  (age,  occupa- 
tion,  years  of  formal  schooling,  mother-tongue),  followed  by  the  actual  experi- 
ment.  In  both  conditions  the  interviewer  presented  the  respondent  with  the 
pictures  one  by  one  and  asked  questions  in  a semi-structured  fashion  about  (i) 
the  recognition  of  the  objects,  (ii)  the  relationship  between  the  objects  and  (iii) 
the  message  of  the  visual.  Respondents  were  invited  to  comment  on  particul- 
ar  aspects  of  the  visual  if  they  had  not  referred  to  them  in  the  initial  response. 

The  first  ten  low-literate  respondents  were  interviewed  in  the  township  of 
KwaMhlanga.  The  interviews  were  conducted  at  the  house  of  one  of  the  par- 
ticipants,  Ms  Elsie  Mahlangu.  The  pre-interview  briefing  on  HIV/AIDS  took 
the  format  of  a group  discussion.  Since  the  mother  tongue  of  all  the  attendees 
was  IsiNdebele,  but  the  preferred  language  was  English,  both  languages  were 
used,  with  interpretation  between  them  by  a fluent  speaker  of  both.  The  actu- 
al  interviews  took  place  individually.  After  the  first  two  respondents,  the  in- 
terviewer  realised  that  the  respondents  were  much  more  fluent  in  Afrikaans 
than  in  English.  Therefore,  all  the  other  interviews  took  place  in  Afrikaans,  as 
it  saved  time,  and  produced  direct  and  reliable  answers.  The  assistance  of  the 
interpreter  was  sought  whenever  a word  or  phrase  in  either  Afrikaans  or 
IsiNdebele  was  not  understood  by  one  of  the  participants. 

The  other  twenty  low-literate  respondents  were  interviewed  in  Constan- 
tia  Park  and  Waterkloof  Glen.  The  interviews  took  place  by  prior  arrangement 
with  the  home  owner,  and  were  conducted  individually.  Pre-interview  brief- 
ings  and  interviews  were  done  individually.  All  the  interviews  were  conduct- 
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ed  in  Afrikaans,  the  lingua  franca  in  the  eastern  suburbs  of  Pretoria.  All  the 
interviews  were  recorded,  transcribed  and  translated  into  English. 

The  interviews  with  the  twenty  five  literate  respondents  were  conducted 
individually,  using  English  as  a medium  of  communication. 

3.4  Coding  and  analysis  ofdata 

After  conducting  a session  of  interviews  the  responses  were  typed  on  data 
sheets.  A code  book  was  devised  on  the  basis  of  the  research  questions  and 
the  specific  characteristics  of  each  picture  included  in  the  testing  instrument. 
The  following  categories  of  information  were  included  in  the  code  book: 

• column  number  , 1-30  (low-literates);  1-25  (literates) 

• column  label,  e.g.  visl  l,  visl_2,  etc. 

• column  descriptor,  e.g.  in  the  case  of  visl_5  - Able  to  identify  shape  as 
a thought  balloon' 

• value,  e.g.  1,  2 

• value  label,  e.g.  1 = 'yes'  (if  the  respondent  gave  an  appropriate  inter- 
pretation  of  the  pictorial  symbol);  and  2 = 'no'  (if  the  respondent  failed 
to  give  an  appropriate  interpretation  of  the  pictorial  symbol). 

The  variables  (column  labels)  and  values  were  read  into  version  13.0  of  the 
statistical  analysis  program  SPSS,  and  the  data  was  coded  in  order  to  calculate 
frequencies  and  averages. 

The  database  comprised  70  variables  or  questions  (represented  by  the  col- 
umn  labels),  of  which  6 covered  the  socio-demographic  details  of  the  respond- 
ents.  The  remaining  64  questions  were  operationalisations  of  the  research 
questions.  The  following  dimensions  of  the  respondents'  picture  comprehen- 
sion  performance  were  covered: 

• Literal  (analogical)  recognition  of  people  and  objects:  pictures  1 through 
10  (control  questions) 

• Interpretation  of  symbolic-analogical  aspects  of  pictures 

— pictorial  metaphors  (pictures  4 and  5) 

• Interpretation  of  symbolic-abstract  aspects  of  pictures 

— abstract  symbols  (pictures  1,  3,  5,  8a,  8b,  9a,  9b) 

— symbolic-abstract  cartoon-style  conventions:  thought  and  speech 
balloons  (pictures  1 through  5) 
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4.  Discussion  of  findings 

4. 1 InterpretJtion  oí anjlogical  pictures 

Both  literate  and  low-literate  respondents  had  little  difficulty  in  recognizing 
analogical  pictures  with  a literal  meaning  (irrespective  of  art  style):  89%  of  the 
low-literate  respondents  and  82%  of  the  literate  respondents  identified  the 
people/objects  in  the  pictures  correctly.  Compare  the  distribution  in  the  fol- 
lowing  table: 


Table  3:  Literal  (purely  analogical)  recognition  of  objects 


Low-literates 

No.  of  correct 

interpretations 

out  of  30 

% 

Literates 

No.  of  correct 
interpretations 

out  of  25 

% 

Picture  1:  Recognises  female  adult  and  child 
in  main  picture 

24 

80 

23 

92 

Picture  1:  Recognises  man,  woman  and  baby 
in  thought  balloon 

22 

73.3 

22 

88 

Picture  2:  Recognises  the  male  adult  and  boy 

24 

80 

23 

92 

Picture  2:  Recognises  the  condom  in  the 
speech  balloon 

26 

86.7 

23 

92 

Picture  3:  Recognises  young  male  and  female 

27 

90 

23 

92 

Picture  3:  Recognises  the  bed  in  the  thought  balloon 

29 

96.7 

25 

100 

Picture  4:  Recognises  young  male  and  female 

30 

100 

24 

96 

Picture  4:  Recognises  condom  in  speech  balloon 

28 

93.3 

15 

60 

Picture  5:  Recognises  the  adult  male  and  female 

30 

100 

21 

84 

Picture  6:  Recognises  the  syringe 

30 

100 

20 

80 

Picture  6:  Recognises  the  test  tube  with  blood 

11 

36.7 

12 

48 

Picture  6:  Recognises  the  male  and  female 

28 

93.3 

20 

80 

Picture  7a:  Recognises  the  person  as  a male  playing 
with  a ball/playing  soccer 

28 

93.3 

22 

88 

Picture  7b:  Describes  the  person  as  a female  who 
is  sitting  on  a couch,  reading  a book 

29 

96.7 

24 

96 

Picture  8a:  Recognises  the  foods 

30 

100 

19 

76 

Picture  8b:  Recognises  the  foods 

29 

96.7 

18 

72 

Picture  9a:  Recognises  the  bottle  of  liquor 

28 

93.3 

22 

88 

Picture  9b:  Recognises  the  ash  tray  and  cigarette 

30 

100 

13 

52 

Picture  10:  Recognises  the  young  female 

30 

100 

19 

76 

Picture  10:  Recognises  pills/medicine 

23 

76.7 

22 

88 

Averages 


89.3 


82.0 
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The  only  object  not  recognised  or  incorrectly  identified  by  both  low-literates 
(37%)  and  literates  (48%)  was  the  test  tube  with  blood  in  picture  6a.  In  the 
low-literate  group,  9 respondents  did  not  recognize  the  test  tube  with  blood 
at  all.  Other  responses  included  'pills'  (4),  'a  man's  thing'  (2),  'vaccine'  (1),  'the 
things  you  put  on  your  lips'(l),  and  'a  condom'  (1).  The  poor  recognition  can 
probably  be  attributed  to  a lack  of  familiarity  with  the  object.  Few  of  the  re- 
spondents  might  have  gone  for  HlV-testing,  or  have  seen  the  type  of  contain- 
er  in  which  blood  is  transported  to  testing  laboratories. 

4.2  InterpretJtion  ofpictures  with  symbolic-abstract  elements 
Two  types  of  symbolic-abstract  pictures  were  reasonably  well  represented  in 
the  corpus,  namely  callouts  (speech  and  thought  balloons)  containing  pic- 
tures  representing  what  is  said  or  thought  by  the  characters  in  the  main  pic- 
tures,  and  abstract  symbols. 

The  following  criteria  were  applied  to  establish  how  well  speech  and 
thought  balloons  were  understood: 

• correct  interpretation  of  the  relationship  between  the  contents  of  the 
callout  and  the  rest  of  the  picture,  demonstrated  by  using  either  a verb 
of  saying  (say,  talk,  discuss,  teach,  tell,  instruct,  etc.)  or  a verb  or  cogni- 
tion  (think); 

• namirig  the  shape  correctly,  by  using  words  such  as  speech/thought  bal- 
loon,  or  callout. 

Tables  4 and  5 summarize  the  responses  relating  to  these  criteria.  On  average 
both  literate  and  low-literate  respondents  scored  low  on  understanding  the 
relationship  between  a speech  or  a thought  balloon  and  the  persons  whose 
speech  or  thoughts  are  captured  in  these  shapes.  It  could  be  assumed  that 
second  language  speakers  of  English  would  not  know  the  appropriate  terms 
for  these  shapes.  However,  the  researcher  was  surprised  that  such  a large  per- 
centage  of  literate  respondents  was  not  acquainted  with  the  pictorial  conven- 
tion  itself.  Less  than  40%  of  them  gave  appropriate  explanations  for  the  rela- 
tionships  between  the  callouts  and  the  main  pictures.  The  fact  that  67%  of  the 
low  literates  and  80%  of  the  literates  seemed  to  understand  the  relationship 
between  the  speech  balloon  and  the  two  characters  (the  man  and  the  boy)  in 
picture  2,  could  perhaps  be  attributed  to  their  interpretation  of  other  visual 
cues,  namely  the  adult  male's  parted  lips. 

Table  6 gives  an  overvíew  of  the  interpretation  of  abstract  symbols  by  the 
two  groups: 
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Table  4:  Correct  interpretation  of  speech  and  thought  balloons 


Low-literates 

No.  of  correct 

interpretations 

out  of  30 

% 

Literates 

No.  of  correct 

interpretations 
out  of  25 

% 

Picture  1:  The  respondent  uses  a verb  of  cognition 
such  as  thinks,  understands,  etc.  to  describe  the  re- 
lationship  between  main  picture  and  thought  balloon 

3 

10 

13 

52 

Picture  2:  The  respondent  uses  a verb  of  saying 
such  as  say,  talk,  telt,  etc,  to  describe  the  relationship 
between  main  picture  and  speech  balloon 

20 

66.7 

20 

80 

Picture  3:  The  respondent  uses  a verb  of  cognition 
to  describe  the  relationship  between  main  picture 
and  content  of  thought  balloon 

1 

3.3 

9 

24 

Picture  4:  The  respondent  uses  a verb  of  saying  to 
describe  the  relationship  between  main  picture 
and  speech  balloon 

11 

36.7 

6 

24 

Picture  5:  The  respondent  uses  a verb  of  cognition 
to  describe  the  relationship  between  the  main 
picture  and  the  thought  balloon 

4 

13.3 

2 

8 

Averages 

26 

37.6 

Table  5:  Correct  labelling  of  speech  and  thought 

Low-literates 

No.  of  correct 

interpretations 

out  of  30 

balloons 

% 

Literates 

No.  of  correct 

interpretations 

out  of  25 

% 

Picture  1:  Able  to  label  shape  as  a thought  balloon 

1 

3.3 

0 

0 

Picture  2:  Able  to  label  the  shape  as  a 

1 

3.3 

2 

8 

speech  balloon 

Picture  3:  Able  to  label  the  shape  as 

0 

0 

1 

4 

a thought  balloon 

Picture  4:  Able  to  label  shape  as  a speech  balloon 

0 

0 

1 

4 

Picture  6:  Able  to  label  the  shape  as  a 

0 

0 

2 

8 

thought  balloon 

Averages 


1.3 


4.8 
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Table  6:  Correct  interpretation  of  abstract  symbols 


Low-literates 

No.  of  correct 

interpretations 

out  of  30 

% 

Literates 

No.  of  correct 

interpretations 

out  of  25 

% 

Picture  1:  Interpret  content  of  thought  balloon 
correctly  (if  a man  and  a woman  have  intercourse 
they  can  have  a baby) 

4 

13.3 

20 

80 

Picture  1:  Able  to  name  abstract  symbols  (+  and  =) 

1 

3.3 

6 

24 

Picture  3:  Interpret  the  content  of  the  thought 
balloon  appropriately 

15 

50 

21 

84 

Picture  3:  Recognize  the  cross  as  a sign 
of  prohibition 

9 

30 

22 

88 

Picture  5:  Able  to  identify  the  question  mark 
in  the  thought  balloon 

5 

16.7 

6 

24 

Picture  5:  Able  to  relate  the  question  mark  in  the 
thought  balloon  to  the  main  picture 

0 

0 

3 

12 

Picture  8a:  Describe  the  food  as  not  healthy 

18 

60 

24 

96 

Picture  8a:  Indicate  the  red  cross  in  support  of 
previous  answer 

13 

43.3 

18 

72 

Picture  8b:  Indicate  the  green  tick  in  support  of 
the  previous  answer 

13 

43.3 

16 

64 

Picture  8b:  Describe  the  food  as  healthy 

22 

73.3 

22 

88 

Picture  9a:  Interpret  the  prohibition  correctly 

27 

90 

23 

92 

Picture  9a:  Indicate  the  red  cross  in  support  of 
previous  answer 

25 

83.3 

22 

88 

Picture  9b:  Interpretthe  prohibition  correctly 

27 

90 

24 

96 

Picture  9b:  Indicate  the  red  cross  in  support  of 
previous  answer 

24 

80 

22 

88 

Averages 

48.3 

71.1 

Table  6 shows  that  the  low-literates  scored  22%  lower  than  the  literates  on  the 
interpretation  of  pictures  with  symbolic-abstract  elements.  This  is  not  surpris- 
ing:  their  lack  of  formal  education  did  not  facilitate  the  development  of  a vo- 
cabulary  for  abstract  symbols,  and  also  resulted  in  a limited  development  of 
formal  (higher  order)  thought  processes  such  as  categorisation  and  critical 
reflection. 

Crosses  and  ticks  were  noticed  and  interpreted  correctly  by  the  low  liter- 
ate  group  in  those  pictures  where  the  symbols  were  emphasised  through  the 
use  of  colour,  and  legibility  (size  and  weight).  A red  cross  as  a sign  of  prohibi- 
tion,  superimposed  on  an  object,  was  recognized  by  approximately  75%  of 
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the  low-literate  respondents  (compare  the  responses  for  pictures  9a  and  9b). 
However,  where  the  abstract  symbol  was  buried  in  the  background,  recogni- 
tion  dropped  by  about  a third.  Less  than  half  of  the  low-literates  recognized 
the  cross  and  the  tick  in  pictures  8a  and  8b.  The  complexity  of  the  picture  also 
seems  to  reduce  the  probability  that  an  abstract-symbolic  visual  will  be  recog- 
nized  and  correctly  interpreted  by  people  with  limited  literacy  skills.  In  pic- 
ture  3 only  a third  of  the  low-literate  respondents  identified  and  correctly 
interpreted  the  cross  in  the  thought  balloon.  Cognitive  load  could  have  been 
decreased  by  introducing  an  additional  cognitive-perceptual  layer,  namely 
colour.  Since  bright  orange  was  used  for  headings  in  the  booklet  from  which 
this  particular  picture  was  taken,  spot  colour  could  have  been  used  for  em- 
phasis  in  black  and  white  line  drawings. 

Mathematical  symbols  presented  major  problems  for  the  low-literate  group. 
Only  one  respondent  was  able  to  name  the  symbols  + and  = in  the  thought 
balloon  of  picture  í,  and  to  give  an  acceptable  interpretation  of  the  content.  A 
confounding  factor  might  have  been  that  the  symbols  in  the  balloon  are  not 
used  in  their  primary  logico-mathematical  senses.  The  symbol  + is  used  as  a 
synonym  for  unite  rather  than  for  the  sum  of,  and  the  symbol  = means  "is  the 
product  of ",  rather  than  "equals".  Another  possible  reason  why  the  + was  not 
interpreted  as  a mathematical  symbol,  is  that  in  the  context  of  health  educa- 
tion,  this  symbol  is  often  used  to  represent  health  care  facilities.  Four  of  the 
low-literate  respondents  identified  the  + as  a sign  for  a hospital  or  clinic. 

4.3  InterpretJtion  o f symbolic-analogical  pictures 

Mayeux  et  al.  (1996)  advise  that  "To  be  effective  with  patients  whose  literacy 
skills  are  low,  patient  education  materials  should  be  short  and  simple,  contain 
culturally  sensitive  graphics  and  encourage  desired  behaviour."  Culturally 
sensitive  graphics  would,  for  instance,  include  pictures  portraying  taboo  ob- 
jects,  and  unfamiliar  pictorial  metaphors.  Although  only  two  culture-depend- 
ent  metaphorical  pictures  were  included  in  the  testing  materials,  the  respon- 
ses  indicated  that  this  kind  of  visual  was  problematic.  Even  the  literate  re- 
spondents  (the  majority  of  whom  are  speakers  of  African  languages)  had  some 
difficulty  in  relating  the  pictorial  metaphor  to  its  intended  meaning  in  picture 
5.  Compare  Table  7: 
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Table  7:  Correct  interpretation  of  metaphor 


Low-literates 

Literates 

No.  of  correct 

No.  of  correct 

interpretations 

interpretations 

out  of  30 

% 

out  of  25 

% 

Picture  4:  Associate  hearts  with  love 

9 

30 

17 

68 

Picture  5:  Recognize  the  red  object  in  the  thought 

9 

30 

4 

16 

balloon  as  a metaphor  for  illness 


Averages  30  42 


When  including  picture  4,  the  researcher  did  not  anticipate  that  the  hearts 
surrounding  the  boy's  head  would  be  ambiguous.  It  was  taken  for  granted 
that  the  romantic  meaning  of  red  hearts  had  been  popularized  by  mass  media 
such  as  cartoons,  greeting  cards,  etc.  The  responses  of  the  low-literate  group 
could  be  interpreted  as  a lack  of  exposure  to  print  materials  in  general.  Anoth- 
er  possible  explanation  for  the  comprehension  problems  that  occurred,  is  the 
influence  of  language-supported  cultural  meanings.  When  5 low-literate  re- 
spondents  answered  that  the  boy  is  'thinking  with  the  heart'  (and  when  asked 
for  an  explanation  they  said  he  was  worrying  or  ruminating),  specialists  of  a 
nurnber  of  African  languages  were  consulted  to  find  out  whether  these  lan- 
guages  contained  a semi-idiomatic  expression  that  would  be  translated  di- 
rectly  as  'thinking  with  the  heart',  but  meaning  "to  worry".  According  to  a 
lexicographic  practitioner  who  speaks  several  African  languages,  the  Sepedi 
expression  o bolela  ka  pelo,  and  the  IsiZulu  expression  ukhuluma  ngenhliziyo,  can 
be  literally  translated  as  'to  talk  with  the  heart',  but  actually  means  "to  wor- 
ry".  Although  the  number  of  respondents  who  assigned  this  meaning  was 
relatively  small,  their  responses  indicated  that  metaphorical  portrayals  should 
never  be  used  without  pretesting  them  among  members  of  the  cultural  group(s) 
of  which  the  audience  is  part. 

Picture  5 was  included  because  it  was  expected  that  the  low-literate  re- 
spondents  may  experience  problems  with  interpreting  the  metaphor  for  AIDS. 
Astonishingly,  not  only  the  low-literates  scored  very  low  on  comprehending 
the  picture  in  the  thought  balloon;  the  literates  performed  worse:  whereas 
30%  of  the  low-literate  respondents  related  the  red  monster  to  sickness  (re- 
sponses  included  the  terms  'sickness',  'germ',  'bacterium'  and  'virus'),  only 
16%  of  the  literates  gave  appropriate  answers.  According  to  African  language 
specialists  the  African  languages  do  not  have  a unified  way  of  metaphorising 
AIDS,  and  it  would  therefore  be  extremely  difficult  to  use  one  single  meta- 
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phorical  picture  to  symbolise  the  meaning  of  AIDS  for  speakers  of  all  the  Afri- 
can  languages. 


5.  Conclusion 

5.7  Summary  of  fíndings 

Consistent  with  previous  research,  it  was  found  that  purely  analogical  (literal; 
representative)  pictures  pose  relatively  few  interpretation  problems  (on  aver- 
age  less  than  18%).  The  fact  that  the  low-literates'  recognition  of  objects  was 
slightly  higher  than  that  of  the  literates  (by  7%)  is  probably  due  to  the  pre- 
interview  briefing  of  the  low-literates  on  the  core  concepts  of  HIV  and  AIDS. 

As  far  as  the  interpretation  of  symbolic-analogical  (metaphorical)  pictures 
by  low-literates  is  concerned,  two  problems  are  worth  mentioning: 

• overriding  of  (intended)  popular  mass  media  interpretations  by  cul- 
turally/linguistically  encoded  meanings  (picture  4); 

• literal  (analogical)  interpretation  of  metaphorical  pictures  (picture  5). 

An  unexpected  finding  regarding  the  literate  group  was  that  they  scored  low- 
er  on  recognising  purely  analogical  pictures  than  the  low-literate  group.  Only 
16%  (as  opposed  to  30%  of  the  low-literates)  related  the  red  monster  in  the 
thoughtballoon  (picture  5)  to  HTV/AIDS,  illness  or  a related  concept.  This  could 
be  due  to  the  absence  of  a pre-interview  briefing  session,  the  style  of  the  inter- 
viewer,  and/or  the  time  allowed  by  the  interviewer  to  peruse  each  picture 
before  responding. 

These  findings  suggest  that  in  special-purpose  communication,  such  as 
health  education  materials,  it  is  dangerous  to  use  metaphorical  portrayals  that 
have  not  been  tested  among  members  of  the  target  populations,  irrespective 
of  literacy  level. 

The  research  in  connection  with  symbolic-abstract  pictures  confirmed  that 
people  with  limited  reading  skills  and  limited  exposure  to  written  media  ex- 
perience  major  problems  interpreting  abstract  picture  conventions  such  as 
speech  balloons  and  thought  balloons;  and  symbols  from  systems  of  formal 
logic,  such  as  plusses,  minuses,  crosses  and  ticks.  Interpretation  problems  are 
compounded  where  cognitive  load  is  high,  e.g.  when  the  legibility  is  poor, 
when  the  symbols  form  part  of  a complex  visual,  or  when  they  occur  in  the 
background  of  a picture. 

Contrary  to  expectations  the  literates'  interpretation  of  the  content  of  speech 
and  thought  balloons  were  only  approximately  12%  better  than  that  of  the 
low-literates,  and  they  also  performed  poorly  on  labelling  these  shapes.  They 
did,  however,  understand  symbols  with  logical/mathematical  elements  signif- 
icantly  better  than  the  low-literates.  They  were  also  generally  better  equipped 
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to  reflect  on  their  responses,  such  as  indicating  a symbol  (cross,  tick,  etc.)  as 
the  source  of  a particular  answer  (e.g.  picture  3 and  8b). 

5.2  LimiUtions  ofthe  research 

Due  to  the  qualitative  (and  therefore  exploratory)  nature  of  the  initial  design 
and  the  availability  of  materials,  the  portfolio  of  testing  materials  did  not  in- 
clude  a sufficient  number  of  pictures  in  each  problematic  category  - for  in- 
stanqe  symbolic-analogical  (metaphorical)  pictures.  The  findings  are  therefore 
not  conclusive,  and  need  further  testing  before  generalisations  can  be  made. 

5.3  Suggestions  to  materials  developers 

It  has  become  clear  that  certain  types  of  pictures  are  problematic  across  a wide 
spectrum  of  literacy  levels.  It  is  therefore  suggested  that  simple,  low-literacy 
materials  with  clear  concrete  graphics  be  considered  for  all  patients,  regard- 
less  of  their  reading  skills  (cf.  NWGLH,  1989).  In  the  US  patients  in  both  high 
and  low  socio-economic  and  reading  ability  groups  have  indicated  that  they 
prefer  short,  simple  and  colourful  materials  (NWGLH,  1989).  A study  compar- 
ing  a simplified  sixth  grade  reading  level  brochure  about  polio  (which  com- 
bined  simple  text  and  line-drawings  with  captions  to  a 10th  grade  reading  level 
brochure)  demonstrated  that  patients  of  all  reading  levels  and  all  socioeco- 
nomic  levels  preferred  the  shorter  arid  simpler  pamphlet.  High-level  readers 
understood  both  brochures  equally  well  but  took  less  time  to  read  the  shorter 
one.  No  one  was  offended  by  its  simplicity. 

Designers  of  public  health  education  materials  in  South  Africa  are  advised 
to  design  these  docurnents  at  a 5th  grade  level  (NWGLH,  1989),  and  include 
colourful,  culturally  relevant  pictures  as  well  as  ample  white  space.  Pictorial 
metaphors,  art  styles  that  distort  objects,  complex  pictures  with  partially  sym- 
bolic  content,  as  well  as  abstract  symbols  from  the  written  language  should  be 
omitted  where  possible. 
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The  assessment  of  entry-level  students'  academic  literacy:  does  it  matter? 

In  Higher  Education  both  nationally  and  internationally,  the  need  to  assess 
incoming  students'  readiness  to  cope  with  the  typical  reading  and  writing 
demands  they  willface  in  the  language-of-instruction  of  their  desired  place  of 
study  is  (almost)  common  cause.  This  readiness  to  cope  with  reading  and 
writmg  demands  in  a generic  sense  is  at  the  heart  ofwhat  is  meant  by  notions 
of  academic  literacy.  'Academic  literacy'  suggests,  at  least,  that  entry-level 
students  possess  some  basic  understanding  of-  or  capacity  to  acquire  an 
understanding  of-  what  it  means  to  readfor  meaning  and  argument;  to  pay 
attention  to  the  structure  and  organisation  of  text;  to  be  active  and  critical 
readers;  and  toformulate  written  responses  to  academic  tasks  that  are  charac- 
terised  by  logical  organisation,  coherence  and  precision  of  expression.  This 
paper  attempts  to  address  two  crucial  questions  in  the  assessment  of  students' 
academic  literacy:  (1)  Does  such  an  assessment  matter,  i.e.  does  understanding 
students’  academic  literacy  levels  have  consequence  for  teaching  and  learning, 
andfor  the  academic  performance  of  students,  in  Higher  Education?  (2)  Do 
generic  levels  of  academic  literacy  in  the  sense  described  above  relate  to  academ- 
ic  performance  in  discipline-specific  contexts?  Attempts  to  address  these  two 
questions  draw  on  comparative  data  based  on  an  assessment  of  students’ 
academic  literacy  and  subsequent  academic  performance  across  two  disciplines 
at  the  University  of  Cape  Town  and  the  Cape  Peninsula  University  of  Technolo- 
gy.  Quantitative  analyses  illustrate  relationships  between  students'  academic 
literacy  levels  and  the  impacts  these  have  on  academic  performance.  Conclu- 
sions  to  the  paper  attempt  a critical  assessment  ofwhat  the  analyses  tell  us 
about  students'  levels  of  academic  literacy;  what  these  levels  ofliteracy  might 
meanfor  students  and  their  teachers;  and  what  the  strengths  and  limitations  of 
assessing  academic  literacy  using  a generic  test  might  be. 
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1 . Introduction 

Internationally  and  nationally,  there  is  a substantial  and  arguably  growing 
interest  in  the  importance  of  assessing  applicants  seeking  study  places  in  High- 
er  Education  using  multiple  rather  than  single  assessment  criteria  (see,  for 
example,  Arce-Ferrer  & Castillo,  2006;  Clemans,  Lunneborg  & Raju,  2004;  Clif- 
fordson,  2006;  Houston,  Knox  & Rimmer,  2007;  Shivpuri,  Schmitt,  Oswald  & 
Kim,  2006;  Stricker,  2004).  It  would  appear  that  this  interest  is  driven  by  a 
number  of  factors  or  forces  that  seem  universal  in  Higher  Education  contexts: 
(1)  a growing  concern  internationally  that  applicants  appear  increasingly  poor- 
ly-prepared  to  cope  with  the  generic  academic  reading,  writing  and  thinking 
demands  placed  upon  them  on  entry  to  Higher  Education  study;  (2)  a con- 
cern  that  the  results  of  conventional  school-leaving  examinations  are  not  nec- 
essarily  providing  interpretable  understandings  of  the  academic  competence 
levels  of  incoming  students;  (3)  international  trends  towards  greater  diversity 
of  educational  background  and  experience  in  student  intakes  - and  a con- 
comitant  need  for  Higher  Education  to  have  a common  understanding  of  the 
differing  academic  levels  of  students  from  these  diverse  educational  back- 
grounds;  (4)  a growing  need  for  Higher  Education  to  be  responsive  to  the 
educational  backgrounds  of  students  in  a learning  and  teaching  sense,  and 
for  an  assessment  of  academic  'needs'  to  be  an  important  first  step  towards 
the  placement  of  students  in  appropriate  curricula  according  to  their  educa- 
tional  background. 

Research  studies  of  the  kind  mentioned  above  underscore  an  increasing 
focus  on  the  use  of  what  are  variously  called  admissions  tests,  entrance  tests 
or  selections  tests  as  a means  of  collecting  information,  about  Higher  Educa- 
tion  applicants,  that  is  complementary  to  conventional  or  traditional  assess- 
ment  measures  such  as  school-leaving  examinations.  There  is  now  more  than 
anecdotal  evidence  that  Higher  Education  institutions  and  admissions  com- 
mittees  or  panels  are  taking  seriously  the  need  for  responsible,  ethical  and 
equitable  approaches  to  admissions  decisions,  and  a parallel  need  to  make 
use  of  the  multiple  sources  of  information  collected  about  applicants  for  the 
placement  of  those  eventually  registered  into  appropriate  curricula.  Further- 
more,  there  is  a clear  need  to  assess  the  outcomes  of  the  use  of  multiple  selec- 
tions  criteria  on  the  academic  progression  of  students  thus  selected. 

The  use  of  admissions  tests  such  as  the  Scholastic  Aptitude  Test  (SAT)  (ETS, 
2007)  or  the  Graduate  Management  Admission  Test  (GMAT)  (GMAC,  2007),  or 
assessments  of  language  proficiency  such  as  the  International  English  Lan- 
guage  Testing  System  (IELTS)  (British  Council,  2007)  or  the  Test  of  English  as  a 
Foreign  Language  (TOEFL)  (ETS,  2007),  have  become  common  cause  in  as- 
sessing  applicants'  readiness  for  Higher  Education.  These  tests  are  implement- 
ed  principally  because  it  is  believed  that  they  will  yield  information  about 
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applicants'  abilities  to  cope  with  the  typical  reading,  writing  and  thinking 
demands  they  will  likely  face  in  Higher  Education  or  that  they  will  indicate 
the  extent  to  which  applicants  will  be  able  to  cope  with  the  language  demands 
placed  upon  them  in  a particular  medium-of-instruction. 

One  form  of  complementary  assessment  that  has  become  commonly  used 
in  the  South  African  Higher  Education  landscape,  to  assess  students'  readi- 
ness  to  cope  with  typical  language  and  learning  demands,  is  an  assessment  of 
academic  literacy.  The  Placement  Test  in  English  for  Educational  Purposes 
(PTEEP)  (AARI)  2007),  for  example,  is  a test  that  is  used  at  the  pre-admissions 
stage  to  assess  applicants'  responsiveness  to  progressively  more  demanding 
reading  and  writing  tasks.  Based  on  the  levels  of  responsiveness  applicants 
demonstrate  in  this  assessment  - when  compared  with  other  applicants  from 
similar  educational  backgrounds  - these  applicants  are  regarded  as  'recom- 
mendable'  or  'not  recommendable'  for  particular  forms  of  Higher  Education 
provision  (for  example,  conventional/mainstream  or  foundational  provision). 
Scores  on  the  PTEEP  are  used  as  complementary  measures  to  other  standard 
or  alternate  assessments  of  applicants'  readiness  for  Higher  Education.  An- 
other  example  of  an  academic  literacy  assessment  that  is  used  in  South  Africa 
is  the  Standardised  Assessment  Test  for  Access  and  Placement  (SATAP)  (SATAR 
2007).  This  assessment  has  historically  been  used  after  applicants  have  been 
selected  for  Higher  Education  studies,  as  a means  of  assessing  the  extent  to 
which  these  students  might  require  assistance  with  coping  with  the  academic 
literacy  demands  they  are  likely  to  face.  A third  and  final  example  of  an  aca- 
demic  literacy  test  is  the  Test  of  Academic  Literacy  Levels  (TALL)  (UAL,  2007), 
used  post-selection  by  a number  of  Higher  Education  institutions  to  place 
students  scoring  below  a certain  score  in  courses  that  teach  generic  academic 
literacy  in  one  or  two  media-of-instruction  (Weideman,  2003;  Van  der  Slik  & 
Weideman,  2005). 

Before  proceeding  any  further  with  the  arguments  in  this  paper,  it  seems 
necessary  to  explain  what  is  meant  by  'academic  literacy'  in  this  context.  Aca- 
demic  literacy  in  the  sense  delineated  in  this  paper  (cf.  Bachman  & Palmer, 
1996;  Yeld,  2001;  Cliff  & Yeld,  2006;  Weideman,  2006)  means  the  extent  to  which 
students  are  able  to: 

• make  meaning  from  texts  that  they  are  likely  to  encounter  in  their  stud- 
ies; 

• understand  words  and  discourse  signals  in  their  contexts; 

• identify  and  track  academic  argument; 

• understand  and  evaluate  the  evidential  basis  of  argument; 

• extrapolate  and  draw  inferences  and  conclusions  from  what  is  stated  or 
given; 
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• identify  main  from  supporting  ideas  in  the  overall  organisation  of  a text; 

• understand  information  presented  visually  (e.g.  graphs,  tables,  flow- 
charts); 

• understand  basic  numerical  concepts  and  information  used  in  text,  in- 
cluding  basic  numerical  manipulations 

In  essence  then,  academic  literacy  is  necessarily  a form  of  verbal  reasoning 
that  is  crucially  language-dependent.  It  follows  that  an  assessment  of  academic 
literacy  becomes  an  assessment  of  a student's  verbal  reasoning  capacity  in  a 
particular  language  and  this,  in  turn,  implies  that  academic  literacy  is  associ- 
ated  with  Higher  Education  medium-of-instruction.  The  assessment  of  aca- 
demic  literacy  is,  therefore,  not  an  assessment  of  language  per  se,  but  an  as- 
sessment  of  the  use  of  language  as  a vehicle  for  making  meaning,  making 
argument  and  understanding  underlying  point. 

The  notion  of  academic  literacy  is  conceived  of  from  a different  but  related 
angle  in  student  learning  research.  There,  it  is  taken  to  refer  to  students'  abil- 
ities  to  process  information  they  read  in  a 'deep'  sense,  i.e.  to  understand  the 
underlying  point,  structure  or  argument  of  what  is  read  (Marton  & Sáljó,  1976a 
& b)  rather  than  to  view  what  is  read  as  consisting  of  isolated  or  discrete  'bits' 
of  information.  Theoretical  frameworks  drawn  from  research  into  how  stu- 
dents  learn  in  Higher  Education  have  drawn  a distinction  between  what  are 
referred  to  as  the  'deep'  and  'surface'  approach  to  learning  (cf.  Marton  & Sáljó, 
1984).  The  former  depicts  learning  - and,  by  extension,  academic  literacy  - as 
a process  of  seeing  how  knowledge  contributes  to  understanding,  how  un- 
derstanding  contributes  towards  'seeing7  the  world  differently,  and  how  'see- 
ing'  differently  leads  to  the  development  of  one's  own  point  of  view  that  is 
based  on  new  understandings  and  insights.  'Surface'  learning,  on  the  other 
hand,  is  described  as  viewing  knowledge  as  having  a discrete,  factual  charac- 
ter,  disconnected  and  disembodied  from  other  knowledge  and  to  be  repro- 
duced  in  an  untransformed  form,  usually  to  fulfil  narrow  assessment  require- 
ments.  It  follows  - from  earlier  discussion  of  the  meaning  of  academic  literacy 
- that  'deep'  approaches  to  learning  appear  close  to  the  reading,  writing  and 
thinking  approaches  described  earlier,  when  the  notion  of  academic  literacy 
was  explicated. 

Later  formulations  drawn  from  student  learning  research  in  Higher  Edu- 
cation  argued  that  'deep'  and  'surface'  approaches  to  learning  are  influenced 
by  the  context  in  which  students  learn  and  by  students'  perceptions  of  that 
context;  by  how  students  conceive  of  what  learning  is;  and  by  students'  un- 
derlying  forms  of  motivation  for  learning.  These,  in  turn,  influence  how  stu- 
dents  approach  typical  reading,  writing  and  thinking  tasks  in  Higher  Educa- 
tion.  The  terms  'meaning'  and  'reproducing'  orchestration  (Meyer,  1991)  were 
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formulated  to  describe  qualitative  distinctions  in  the  ways  in  which  individu- 
al  students  'orchestrated'  their  learning  in  accordance  with  their  conceptions 
of  learning,  their  perceptions  of  their  learning  contexts  and  their  complex  or 
less  complex  understandings  of  the  purposes  of  Higher  Education  study. 

Whether  one  approaches  notions  of  academic  literacy  as  being  about  cop- 
ing  with  typical  academic  reading  and  writing  demands  or  whether  one  views 
academic  literacy  as  being  related  to  'deep'  approaches  or  'meaning7  orches- 
trations  towards  learning,  the  commonality  in  the  two  approaches  appears  to 
lie  in  language.  Students  approach  language  to  read  texts  in  Higher  Educa- 
tion;  they  use  language  to  express  their  understandings  of  these  texts  or  to 
produce  viewpoints  of  their  own;  and  they  are  exposed  to  or  use  language  to 
engage  with  their  own  understandings  of  what  learning  is.  Assessments  of 
academic  literacy,  then,  are  assessments  of  responses  to  and  production  of 
language,  albeit  language  in  a specialised,  applied  form:  the  language  of  rea- 
soning,  argument,  exposition,  explanation. 

As  was  argued  earlier,  a common  assumption  about  assessing  students' 
levels  of  academic  literacy  - whether  prior  to  or  on  entry  to  Higher  Education 
— is  that  important  understandings  about  how  these  students  will  cope  will  be 
gained  from  this  assessment  process.  A second  assumption  appears  to  be  that 
assessments  of  generic  forms  of  academic  literacy  are  useful  for  understand- 
ing  how  students  will  cope  in  discipline-specific  contexts.  A third  - perhaps 
less  common  - assumption  is  that  assessments  of  academic  literacy  will  lead 
to  specific  learning  and  teaching  interventions  to  support  those  students  who 
are  deemed  not  yet  ready  to  cope  with  the  generic  academic  literacy  demands 
placed  upon  them  in  Higher  Education. 

This  takes  us  back  to  the  title  of  this  paper:  does  the  assessment  of  stu- 
dents'  academic  literacy  matter,  i.e.  in  this  context,  (1)  are  results  on  tests  of 
academic  literacy  associated  with  subsequent  academic  performance  of  stu- 
dents  across  a range  of  disciplines  and,  if  so,  how;  and  (2)  do  results  on  tests  of 
academic  literacy  provide  useful  information  in  suggesting  teaching  and  learn- 
ing  interventions  necessary  to  improve  students'  levels  of  academic  literacy, 
and  with  what  consequence  for  student  achievement?  There  is  ready  intui- 
tive  and  intellectual  understanding  amongst  Higher  Education  academics  that 
the  characteristics  of  academic  literacy  as  defined  earlier  in  this  paper  are  im- 
portant  if  students  are  going  to  become  academically  literate  in  the  generic 
sense,  but  there  is  arguably  less  agreement  about  how  important  this  generic 
academic  literacy  is  in  discipline-specific  contexts. 

The  studies  described  and  analysed  later  in  this  paper  represent  an  attempt 
to  respond  to  questions  raised  in  the  previous  paragraph  by  being  focused  on 
(1)  the  predictive  validity  of  a generic  test  of  academic  literacy;  and  (2)  the 
consequences  for  teaching  and  learning  of  data  collected  from  this  generic 
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test.  The  studies  are  based  on  the  use  of  the  PTEEP  (referred  to  earlier),  and  it 
is  to  an  explication  of  the  construct  and  psychometric  properties  of  the  PTEEP 
that  this  paper  now  turns. 


2.  An  academic  literacy  test:  the  Placement  Test  in  English  for  Educational 
Purposes  (PTEEP) 

The  construct  or  'blueprint'  for  the  PTEEP  is  fully  explicated  in  Yeld  (2001) 
and  a detailed  explanation  is  not  attempted  here.  The  principal  features  of  the 
approach  to  the  development  of  the  PTEEP  are  that  it  is:  (1)  a generic  test, 
designed  to  provide  complementary  information  to  traditional  achievement 
tests  (such  as  the  school-leavingexamination);  (2)  developed  by  national  inter- 
disciplinary  teams  of  expertise,  to  increase  both  its  face  and  content  validity; 
(3)  relatively  curriculum-independent,  so  as  to  downplay  the  role  of  prior  ex- 
posure  to  knowledge;  (4)  designed  to  assess  language  as  a vehicle  for  academ- 
ic  study  and  reasoning  rather  than  language  per  se;  (5)  developed  according 
to  a theme  and  a set  of  specifications,  so  as  to  ensure  that  engagement  for  the 
writers  can  be  'scaffolded',  made  progressively  more  complex,  and  be  authen- 
tic  to  a Higher  Education  context  (adapted  from  Cliff,  Hanslo,  Ramaboa  & 
Visser,  2005). 

Table  1 shows  the  PTEEP  construct  operationalised  in  the  form  of  a set  of 
specifications  that  depict  the  reasoning  approaches  assessed  in  the  test  (Cliff, 
Yeld  & Hanslo,  2003  - adapted  from  Yeld,  2001;  Bachman  & Palmer,  1996). 

As  can  be  seen  from  Table  1,  the  construct  of  the  PTEEP  is  conceptually 
constituted  of  nine  sub-constructs  that  cover  reasoning  and  meaning-making 
at  a word,  sentence,  paragraph  and  argument  level.  An  important  feature  of 
the  PTEEP  is  its  additional  focus  on  visual  and  numerical  literacy:  these  sub- 
constructs  are  included  in  the  PTEEP  because  they  contain  special  forms  of 
language  that  are  central  components  of  most,  if  not  all,  academic  programmes 
of  instruction. 

The  2007  PTEEP  has  an  overall  Cronbach  alpha  reliability  of  0.89  (typically, 
overall  reliabilities  for  the  test  are  between  0.85  and  0.92)  - if  the  edit-type 
question  is  removed  from  the  analysis,  the  alpha  rises  to  0.92.  The  Cronbach 
alpha  is  based  on  a sample  of  n = 2456  writers. 

Table  2 shows  the  coefficients  of  correlation  amongst  the  sub-constructs  of 
the  2007  PTEEE 
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Table  1:  PTEEP  specifications 


Skill  Assessed 

Explanation  of  Skill  Area 

Vocabulary 

Students’  abilities  to  derive/work  out  word  meanings  from  their  context 

Metaphorical  Expression 

Students’  abilities  to  understand  and  work  with  metaphor  in  language. 

This  includes  their  capacity  to  perceive  language  connotation,  word  play, 
ambiguity,  idiomatic  expressions,  and  so  on 

Extrapolation,  application 
and  inferencing 

Students’  capacities  to  draw  conclusions  and  apply  insights,  either  on  the  basis 
of  what  is  stated  in  texts  or  is  implied  by  these  texts. 

Understanding  the 
communicative  function 

of  sentences 

Students’  abilities  to  ’see’  how  parts  of  sentences  / discourse  define  other  parts; 
or  are  examples  of  ideas;  or  are  supports  for  arguments;  or  attempts  to  persuade 

Understanding  relations 
between  parts  of  text 

Students’  capacities  to  ’see’  the  structure  and  organisation  of  discourse  and 
argument,  by  paying  attention  - within  and  between  paragraphs  in  text-  to 
transitions  in  argument;  superordinate  and  subordinate  ideas;  introductions 
and  conclusions;  logical  development 

Understanding  text  genre 

Students’  abilities  to  perceive  'audience’  in  text  and  purpose  in  writing,  including 
an  ability  to  understand  text  register  (formality  / informality)  and 
tone  (didactic/informative/persuasive/etc.) 

Separating  the  essential 
from  the  non-essential 

Students’  capacities  to  'see’  main  ideas  and  supporting  detail;  statements  and 
examples;  facts  and  opinions;  propositions  and  their  arguments; 
being  able  to  classify,  categorise  and  'label’ 

Understanding  information 
presented  visually 

Students’  abilities  to  understand  graphs,  tables,  diagrams,  pictures, 
maps,  flow-charts 

Understanding  basic 
numerical  concepts 

Students’  abilities  to  make  numerical  estimations;  comparisons;  calculate 
percentages  and  fractions;  make  chronological  references  and  sequence 
events/processes;  do  basic  computations 

The  mostly  moderate  correlations  between  the  PTEEP  sub-constructs  suggests 
there  to  be  some  empirical  support  for  the  conceptual  sub-constructs  as  de- 
fined  in  Table  1.  The  sub-constructs  for  the  most  part  seem  to  be  assessing 
aspects  of  academic  literacy  that  are  at  least  partly  discrete  from  one  another, 
which  seems  justification  for  the  separation  of  the  construct  into  its  sub-con- 
structs.  Given  the  large  sample  size  from  which  these  data  were  drawn  (n  = 
2456)  and  the  diversity  of  the  writer  pool  in  terms  of  demographic  factors 
(such  as  school  and  linguistic  background),  correlations  in  Table  2 are  argua- 
bly  between  the  sub-constructs  of  the  test  rather  than  related  to  the  homoge- 
neity  of  the  writer  pool. 

Two  exceptions  are  apparent  from  Table  2:  (1)  the  correlation  of  0.97  be- 
tween  the  'vocabulary'  and  'discourse'  sub-constructs  suggests  that  writer 
performance  in  one  is  strongly  associated  with  writer  performance  in  the  oth- 
er.  This  seems  theoretically  surprising,  but  can  be  explained  by  the  fact  that 
the  questions  assessing  discourse  indicators  in  the  2007  PTEEP  in  many  cases 
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Table  2:  Correlations  amongst  PTEEP  sub-constructs 


Vocab 

Metaph 

Infer 

Relat 

Senten 

Disc 

Genre 

Essent 

Visual 

Numeric 

Vocabulary 

0.56 

0.57 

0.38 

0.44 

0.97 

0.60 

0.74 

0.73 

0.73 

Metaphor 

0.56 

0.46 

0.30 

0.34 

0.56 

0.47 

0.55 

0.49 

0.49 

Inference 

0.57 

0.46 

0.32 

0.34 

0.57 

0.45 

0.55 

0.52 

0.52 

Relations 

0.38 

0.30 

0.32 

0.22 

0.38 

0.33 

0.39 

0.35 

0.35 

Sentences 

0.44 

0.34 

0.34 

0.22 

0.44 

0.35 

0.45 

0.40 

0.40 

Discourse 

0.97 

0.56 

0.57 

0.38 

0.44 

0.61 

0.74 

0.74 

0.74 

Genre 

0.60 

0.47 

0.45 

0.33 

0.35 

0.61 

0.58 

0.54 

0.54 

Essential 

0.74 

0.55 

0.55 

0.39 

0.45 

0.74 

0.58 

0.67 

0.67 

Visual 

0.73 

0.49 

0.52 

0.35 

0.40 

0.74 

0.54 

0.67 

1.00 

Numerical 

0.73 

0.49 

0.52 

0.35 

0.40 

0.74 

0.54 

0.67 

1.00 

Note:  p < 0.05  in  all  cases 

The  sub-constructs  are  defined  in  Table  1 . In  Table  2,  however,  the  'relations’  sub-construct 
has  been  separated  into  two:  ‘relations'  and  'discourse’. 


asked  writers  to  assess  the  meanings  of  words  from  academic  word  lists,  for 
example,  'however';  'nevertheless';  'because';  and  so  on.  The  correlation  of 
1.00  between  the  'visual'  and  the  'numerical'  sub-construct  is  not  surprising, 
since  writer  performance  on  these  two  constructs  was  assessed  by  the  same 
set  of  questions. 

Typically,  the  PTEEP  consists  of  between  65  and  70  items/questions,  divid- 
ed  into  the  following  question-types:  multiple-choice  questions;  short-response 
questions;  a flow-chart/concept  map  question;  an  edit-type  question;  and  a 
one-page  expository  essay  question.  There  are  at  least  three  texts  for  reading 
in  the  PTEEI^  all  of  which  are  related  to  the  theme  for  that  particular  test. 

Table  3 depicts  correlations  amongst  a number  of  the  question-types  in  the 
2007  PTEEI^  as  well  as  the  correlations  between  these  question-types  and  the 
total  score  of  writers  on  the  test. 


Table  3:  Correlations  amongst  question-types  on  the  PTEEP 


Total 

Short  pieces 

Edit  Question 

Essay 

Multi-choice 

Total 

0.95 

0.88 

0.84 

0.92 

Short  pieces 

0.95 

0.93 

0.69 

0.83 

Edit  Question 

0.88 

0.93 

0.64 

0.77 

Essay 

0.84 

0.69 

0.64 

0.66 

Multi-choice 

0.92 

0.83 

0.77 

0.66 

Note:  p < 0.05  in  all  cases. 
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The  high  correlations  between  various  question  types  and  the  total  score  of 
writers  on  the  test  suggest  that  assessment  using  any  one  question  type  will 
suffice  for  determining  the  overall  performance  of  writers.  In  particular,  the 
multiple-choice  questions  on  their  own,  or  the  short  response  pieces  on  their 
own,  are  very  strongly  correlated  with  the  total  score.  The  correlation  of  the 
short  pieces  to  the  total  score  is  somewhat  surprising,  given  that  these  pieces 
are  assessed  by  different  markers,  but  it  is  also  encouraging  evidence  of  stand- 
ardisation  amongst  these  markers  for  this  question-type.  The  more  moderate 
correlations  amongst  different  question-types  on  the  test  suggest  that,  although 
any  one  question-type  might  be  useful  for  predicting  overall  writer  perform- 
ance  on  the  test,  each  of  the  question-types  does  yield  somewhat  discrete  in- 
formation  about  writer  performance  - or  that  marker  standardisation,  whilst 
reasonable,  has  not  yet  reached  completely  desirable  levels. 

In  summarising  this  section  of  the  paper  - and  to  return  to  the  topic  of 
whether  an  assessment  of  academic  literacy  as  measured  by  the  PTEEP  mat- 
ters  - it  would  seem  that  there  is  justification  for  the  division  of  the  PTEEP 
construct  into  its  sub-constructs,  but  it  would  also  seem  that  there  is  some 
degree  of  overlap  amongst  the  sub-constructs.  This  is  not  surprising,  given 
that  academic  literacy  would  seem  to  be  a complex  construct  the  sub-con- 
structs  of  which  cannot  wholly  be  separated  into  constituent  parts. 


3.  Associations  between  PTEEP  scores  and  academic  performance 

In  one  very  tangible  sense,  assessment  of  academic  literacy  might  matter:  if 
academic  literacy  can  be  shown  to  have  associations  with  subsequent  aca- 
demic  performance  in  Higher  Education.  This  section  of  the  paper  will 
deal  with  two  approaches  to  explorations  into  associations  between 
PTEEP  and  academic  performance1.  The  first  approach  is  a high-level 
(trend)  exploration  of  the  extent  to  which  scores  on  the  PTEEP  have 
association  with  academic  performance  in  two  contrasting  disciplinary 
contexts,  viz.  Engineering  and  Humanities.  The  second  approach  at  a 
programme-specific  level  assesses  the  relations  between  PTEEP  and  a 
postgraduate  Engineering  studies  context,  and  the  value  of  the  PTEEP 
and  its  construct  for  teaching  and  learning  purposes. 

Figure  1 shows  the  associations  between  PTEEP  scores  (expressed  as  a rank- 
ing  of  students  from  decile  1 - top  decile  - to  decile  10  - bottom  dedle)  and 
mean  academic  performance  for  the  2002  cohort  of  University  of  Cape  Town 
Engineering  students  at  the  end  of  their  first  academic  year  of  study.  For  eas- 
ier  reporting,  decile  rankings  have  been  grouped  in  pairs,  and  for  examining 
trend-level  associations,  mean  academic  performance  score  has  been  computed 
as  a simple  average  of  academic  performance  over  the  courses  taken  by  these 


1 Not  all  data  for  these 

explorations  are  included  in 
this  paper,  for  reasons  of 
brevity.  Full  analyses  are 
available  for  scrutiny  from  the 
first  author. 
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Figure  1:  Associations  between  PTEEP  scores  and  academic  performance  - 
2002  Engineering  students  in  their  first  year  of  studies 


students.  Note  that  the  2002  cohort  of  students  has  been  further  sub-divided 
into  two  gi'oups:  those  students  who  were  registered  for  /mainstream,  (con- 
ventional,  standard  curriculum)  programmes  and  those  registered  for  foun- 
dation  (reduced  or  extended  curriculum)  programmes. 

From  Figure  1,  it  can  be  seen  that  for  mainstream  students,  PTEEP  per- 
formance  is  associated  with  noticeable  'spreads'  of  scores  in  academic  per- 
formance  terms  at  the  endvof  first-year.  The  trend,  though,  for  mainstream 
students  is  that  higher  decile  ranking  on  PTEEP  (particularly  deciles  1 and  2) 
is  associated  with  higher  mean  academic  performance  and  lower  numbers  of 
students  scoring  below  a 50%  mean.  Assessing  academic  literacy  by  means  of 
the  PTEEP  does  appear  to  rnatter  in  academic  performance  terms  for  main 
stream  students  at  the  end  of  first-year. 

For  Foundation  programme  students,  higher  PTEEP  scores  are  not  as  clearly 
related  to  higher  academic  performance  scores  as  they  are  for  mainstream 
students.  There  is  still  a tendency,  though,  for  higher  PTEEP  scores  to  be  asso- 
ciated  with  lower  nurnbers  of  students  scoring  below  a 50%  mean  for  aca- 
demic  performance.  Assessing  the  academic  literacy  of  Foundation  programme 
students  using  the  PTEEP  does  appear  to  matter  in  terms  of  lower  PTEEP 
scores  predicting  the  numbers  of  students  falling  below  50%  mean,  but  mat- 
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2002  Engineering  students  in  second-year 
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Figure  2:  Associations  between  PTEEP  scores  and  academic  performance  - 
2002  Engineering  students  in  their  second  year  of  studies 


ters  less  in  terms  of  higher  PTEEP  scores  relating  to  higher  academic  perform- 
ance  than  it  does  for  mainstream  students. 

Figure  2 shows  associations  between  PTEEP  scores  and  academic  perform- 
ance  for  the  2002  intake  of  Engineering  students  in  their  second  year  of  stud- 
ies.  Essentially,  the  patterns  of  association  are  similar  for  mainstream  and  foun- 
dation  programme  students  as  they  were  for  first-year  performance. 

In  a contrasting  disciplinary  context,  i.e.  Humanities,  associations  between 
PTEEP  scores  and  mean  academic  performance  produce  patterns  of  the  kind 
illustrated  in  Table  4.  The  Table  shows  associations  between  bands  of  PTEEP 
performance  and  mean  academic  performance  at  the  end  of  first-year  for  two 
cohorts  of  Humanities  students,  viz.  the  2004  and  2005  intakes.  'Bands'  of  per- 
formance  refers  to  the  grouping  of  PTEEP  performance  by  deciles  as  indicat- 
ed  in  Table  4. 

From  Table  4,  it  is  clear  that  approximately  70%  of  Deciles  1-3  students  in 
both  years  achieved  mean  second  or  first  class  pass  scores  and  between  ap- 
proximately  65%  and  70%  of  the  Deciles  8-10  students  scored  third  class  pass- 
es  or  failed.  In  Humanities,  higher  ranked  PTEEP  performance  seems  associ- 
ated  with  a higher  level  of  pass;  lower  ranked  PTEEP  performance  associated 
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Table  4:  Associations  between  PTEEP  scores  and  academic  performance  - 
2004  and  2005  Humanities  students  in  their  first  year  of  studies 


Year  Intake  (numbers  of  students  in  each  category) 

2004 

2005 

Deciles  1-3 

Deciles  4-7 

Deciles  8-10 

Deciles  1-3 

Deciles  4-7 

Deciles  8-10 

Fail 

13 

9 

7 

19 

18 

9 

third  class  pass 

64 

86 

41 

81 

90 

62 

second  class  pass 

197 

92 

18 

198 

104 

35 

first  class  pass 

7 

2 

0 

19 

2 

1 

Total 

281 

189 

66 

317 

214 

107 

Year  Intake  (percentage  of  students  in  each  category) 

2004 

2005 

Deciles  1-3 

Deciles  4-7 

Deciles  8-10 

Deciles  1-3 

Deciles  4-7 

Deciles  8-10 

Fail 

4.63% 

4.76% 

10.61% 

5.99% 

8.41% 

8.41% 

third  class  pass 

22.78% 

45.50% 

62.12% 

25.55% 

42.06% 

57.94% 

second  class  pass 

70.11% 

48.68% 

27.27% 

62.46% 

48.60% 

32.71% 

first  class  pass 

2.49% 

1.06% 

0.00% 

5.99% 

0.93% 

0.93% 

Total 

100% 

100% 

100% 

100% 

100% 

100% 

with  a lower  level  of  pass.  Furthermore,  the  mean  academic  performance  lev- 
els  of  the  Deciles  1-3  students  are  statistically  significantly  higher  than  the 
mean  academic  performance  levels  of  both  of  the  other  two  groups.  This  sug- 
gests  that  in  an  environment  of  competition  for  academic  places  in  this  Facul- 
ty,  the  Deciles  1-3  students  would  be  more  likely  to  be  academically  successful 
(albeit  that  data  is  limited  here  to  the  first  year  of  study).  The  two  studies 
above  represent  investigations  conducted  using  a trend  level  approach  to  as- 
sessing  associations  between  PTEEP  and  subsequent  academic  performance. 

The  third  study  described  below  represents  an  attempt  to  explore  the  im- 
pact  of  teaching  and  learning  on  student  performance  on  the  PTEEP  The  con- 
text  for  the  study  was  a postgraduate  Engineering  course  in  Project  Manage- 
ment,  where  students  wrote  the  PTEEP  at  the  commencement  of  their  studies 
and  again  at  the  conclusion  of  their  study  programme.  The  principal  aim  of 
this  process  was  to  assess  the  extent  to  which  students'  academic  literacy  as- 
sessed  by  the  PTEEP  could  be  said  to  have  altered  or  remained  stable  after  a 
programme  of  study,  i.e.  did  'good'  or  'poor'  performance  on  the  PTEEP  re- 
main  stable  or  improve  at  the  second  administration  of  the  test?  The  second 
aim  of  this  study  was  to  explore  the  extent  to  which  the  PTEEP  could  be  used 
to  identify  academic  literacy  strengths  and  weaknesses  in  a group  of  students, 
i.e.  could  performance  on  the  test  be  used  to  guide  teaching  and  learning? 
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Table  5:  Comparison  of  mean  PTEEP  performance  of  postgraduate 
Engineering  students  on  two  separate  occasions 


Mean  PTEEP  Score 

First  Occasion 

Mean  PTEEP  Score 

Second  Occasion 

Full  cohort 

45.5% 

45.7% 

Sub-cohort  who  scored  below  30%  on  the 

first  occasion 

23.8% 

27% 

Sub-cohort  who  scored  between  31%  and  50% 

on  the  first  occasion 

40.7% 

40.4% 

Sub-cohort  who  scored  above  50%  on  the 

first  occasion 

56% 

55% 

Table  5 shows  the  differences  in  mean  PTEEP  performance  for  the  2005 
cohort  of  postgraduate  Engineering  students  who  wrote  the  test  on  two  occa- 
sions  in  the  academic  year. 

As  will  be  noted  from  Table  5,  mean  PTEEP  performance  remained  rela- 
tively  stable  from  one  test  administration  to  the  next  - differences  in  mean 
PTEEP  performance  were  not  statistically  significant.  The  only  sub-group  for 
whom  differences  (impróvements)  in  performance  could  be  seen  were  the 
group  whose  PTEEP  performance  had  been  weakest  on  the  first  administra- 
tion  occasion.  Stabie  rnean  PTEEP  performance  for  the  sub-cohort  who  scored 
above  50%  on  the  first  occasion  is  arguably  acceptable:  these  students  per- 
formed  creditably  on  the  first  occasion,  and  retained  that  level  of  perform- 
ance.  Stability  or  minor  improvement  in  the  other  two  sub-cohorts  is  some- 
what  worrying.  The  weakest  sub-cohort  did  show  some  improvement  in  mean 
PTEEP  performance  (to  27%),  but  from  a poor  initial  performance  base. 

There  may  be  a nurnber  of  possible  explanations  for  the  lack  of  improve- 
ment  in  PTEEP  scores  for  the  weaker  sub-cohorts:  (1)  for  students  weak  in 
academic  literacy,  one  year  is  not  sufficient  to  improve  this  academic  literacy 
in  a teaching  context  that  is  not  explicitly  designed  to  address  academic  liter- 
acy  as  defined  in  this  paper;  (2)  student  motivation  to  demonstrate  improve- 
ment  in  a generic  academic  literacy  test  is  low  if  these  students  can  see  no 
apparent  relationship  between  what  is  assessed  in  this  test  and  what  is  as- 
sessed  in  a discipline-specific  context  such  as  this  postgraduate  Engineering 
one.  The  most  compelling  explanation  for  the  lack  of  improvement  lies  in  the 
absence  of  explicit  intervention  of  the  academic  literacy  kind  assessed  by  the 
PTEEP  in  the  programme  of  teaching  and  learning  these  students  were  regis- 
tered  for.  Conventional  coursework  per  se  proved  insufficient  to  change  their 
scores  on  an  academic  literacy  test. 

Particular  approaches  to  academic  reading,  writing  and  thinking  that  ap- 
peared  to  be  weakest  for  the  group  of  students  as  a whole  (data  available  from 
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these  authors)  were:  (1)  metaphorical  expression  - students'  capacity  to  un- 
derstand  and  use  analogous,  "pictorial"  and  non-literal  language  and  reason- 
ing;  (2)  text  genre  - students'  capacity  to  understand  that  writers  have  differ- 
ent  "audiences"  and  purposes  for  writing,  and  that  these  influence  what  and 
how  they  write;  (3)  own  voice  - students'  capacity  to  produce  their  own  log- 
ical  argument,  structure  this  argument  and  use  appropriate  language  in  its 
formulation.  However,  these  weaknesses  in  an  academic  literacy  sense  were 
not  explïcitly  addressed  in  the  teaching  programme.  Course  lecturers  did  not 
engage  with  the  discipline-specific  meanings  and  consequences  of,  for  exam- 
ple,  students'  test  weaknesses  in  analogous  reasoning,  text  genre,  or  capacity 
to  produce  structured  argument.  Nor  was  course  assessment  in  the  postgrad- 
uate  Engineering  context  explicitly  related  to  the  assessment  embodied  in  the 
academic  literacy  test.  So  it  may  be  that  assessing  students'  academic  literacy 
for  learning  improvement  does  not  necessarily  'matter'  - unless  this  assess- 
ment  is  tied  to  direct  teaching  interventions  aimed  at  addressing  weaknesses 
identified. 


4.  Concluding  discussion 

We  return  to  the  title  of  our  paper  and  consider  again  whether  an  assessment 
of  the  academic  literacy  of  entry-level  Higher  Education  students  matters.  We 
have  explored  the  notion  of  what  is  rneant  by  'matters'  at  a number  of  levels 
in  this  paper.  Firstly,  we  have  considered  the  extent  to  which  an  assessment  of 
generic  academic  literacy,  such  as  the  PTEEE  is  regarded  by  Higher  Education 
academics  as  having  validity,  i.e.  we  have  considered  the  face  validity  of  the 
PTEEP,  and  have  observed  that  the  theoretical  grounding  of  the  construct  of 
the  PTEEP  in  international  studies  of  language  assessment  and  of  student  learn- 
ing  helps  to  establish  this  validity.  We  have  also  noted  that  the  participation 
by  interdisciplinary  national  teams  in  the  development  and  operationalisa- 
tion  of  the  construct  of  the  PTEEP  further  assists  in  establishing  both  face  and, 
in  so  far  as  this  is  systematically  considered  and  articulated,  also  content  va- 
lidity.  At  an  empirical  level,  we  have  reported  on  the  reliability  of  the  PTEEP 
and  the  coherence  of  the  construct  and  its  sub-constructs.  We  have  argued 
that  there  appears  to  be  some  empirical  support  for  the  division  of  the  con- 
struct  into  its  constituent  parts,  but  that  there  also  appears  to  be  some  degree 
of  overlap  amongst  the  constituents.  We  have  also  presented  evidence  that 
some  question-types  on  the  test  might  of  themselves  be  sufficient  to  assess 
students'  academic  literacy,  but  that  there  are  grounds  for  arguing  that  read- 
ing-response  type  questions  (multi-choice  questions)  assess  different  kinds  of 
academic  literacy  to  writing-response  type  questions  (productive  elements  in 
the  PTEEP). 
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Secondly,  we  have  assessed  the  extent  to  which  assessments  such  as  the 
PTEEP  'matter'  in  terms  of  their  having  associations  with  subsequent  student 
academic  performance.  Large-scale  studies  of  the  kind  described  in  Engineer- 
ing  and  Humanities  contexts  in  this  paper  suggest  that  differing  levels  of  per- 
formance  on  the  PTEEP  are  associated  with  differing  levels  of  academic  per- 
formance  across  both  mainstream  and  foundation  programme  provision.  In 
the  mainstream  context,  higher  scores  on  the  PTEEP  appear  to  be  associated 
with  academic  performance  scores  and  lower  scores  on  the  PTEEP  with  lower 
academic  performance.  In  the  foundation  programme  context,  lower  scores 
on  the  PTEEP  appear  to  have  some  association  with  lower  scores  academical- 
ly.  Higher  scores  on  the  PTEEP  are  less  associated  with  higher  academic  per- 
formance  scores  than  they  were  for  mainstream  students,  but  are  more  likely 
to  be  predictive  of  success  than  failure  for  foundation  programme  students. 

Smaller-scale  studies  of  the  kind  reported  on  in  the  postgraduate  Engi- 
neering  context,  where  explorations  of  a direct  relationship  between  PTEEP 
and  academic  performance  were  attempted,  provide  no  significant  evidence 
that  PTEEP  scores  improve  after  a period  of  academic  study.  At  face-value, 
however,  there  would  seem  to  be  evidence  of  improvement  in  PTEEP  per- 
formance  for  those  students  who  performed  poorly  on  the  PTEEP  at  the  first 
time  of  writing.  We  conclude  that  PTEEP  performance  may  not  'matter'  un- 
less  it  is  explicitly  addressed  in  the  context  of  discipline-specific  curricula  and 
unless  the  academic  literacy  assessed  in  the  PTEEP  is  integrated  into  the  teach- 
ing,  learning  and  assessment  of  the  disciplinary  programme. 
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Functional  multilingualism  at  the  North-West  University: 

Communication  difficulties  in  meetings 

Tlie  new  North-West  University  (NWU)  came  into  existence  on  1 January 
2004.  The  university  has  campuses  in  Mafikeng,  Potchefstroom  and  in  the  Vaal 
Triangle  (in  Vanderbijlpark),  and  the  institutional  head  office  is  in  Potchef- 
stroom.  The  language  contexts  of  the  campuses  differ  substantially,  and  to 
accommodate  this  reality,  the  institution  committed  itself  to  functional  multi- 
lingualism  very  early  in  the  merging  process.  In  2007,  Council  approved  a 
functional  multilingual  language  policy  and  a language  plan  that  stipulates 
how  functional  multilingualism  is  implemented  in  the  domains  of  teaching  and 
learning,  research,  the  organized  student  life  and  administration. 

In  this  article  I would  like  tofocus  on  the  language  challenges  faced  by  the 
administrators  at  the  NWU.  The  theoretical  framework  ofthe  study  is  that  of 
the  ethnography  of  communication  (Saville-Troike,  2003).  Research  questions  of 
interest  in  this  article  are:  (a)  What  are  the  experiences,  perceptions  and 
attitudes  of  administrative  workers  at  the  NWU  towards  the  implementation  of 
the  functional  multïlingual  language  policy  and  plan?  (b)  What  could  be 
learned  from  these  experiences,  perceptions  and  attitudes  to  improve  the 
implementation  of  the  functional  multilingual  language  policy  and  plan  at  the 
NWU?  The  attempt  of  the  mstitution  at  the  implementation  of  functional 
multilingualism  is  unique  in  the  higher  education  context  in  South  Africa  and 
it  is  argued  that  this  courageous  effort  ofthe  institution  should  be  studied  in  a 
qualitative  and  longitudinal  manner  because  it  could  provide  insights  into  the 
way  forward  in  the  post-1994  South  Africa  where  there  is  a struggle  in  different 
domains  to  come  to  terms  with  its  multilingual  context. 

The  main  findings  of  the  article  are  that  the  complexity  of  multilingualism  and 
the  identities  of  multilingual  language  users  should  be  considered  carefully  in 
the  implementation  process  of  a functional  multilingual  language  policy  and 
plan.  The  realities  of  the  multilingual  identities  ofla?iguage  users  challenge 
seemingly  "obvious"  solutions  (like  the  use  of  interpretation  services  at  meet- 
ings)  often  used  in  multilingual  contexts.  A second  finding  is  that  the  use  of  an 
ethnographic  approach  seems  productive  to  unearth  attitudes  a?id  perceptions  of 
administrators  struggling  to  work  effectively  i?i  the  context  ofthis  multilingual 
workplace  which  would  not  have  been  learned  if  more  "qua?ititative"  research 
?nethods  were  used  exclusively. 
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1 . Contextualisation  and  probiem  statement 

The  NWU  is  striving  to  manage  the  multilingual  nature  of  its  staff  and  stu- 
dents  by  establishing  a language  policy  premised  on  functional  multilingual- 
ism  for  the  domains  of  teaching  and  learning,  research,  the  organised  student 
life  and  administration.  The  language  policy  statement  related  to  the  imple- 
mentation  of  functional  multilingualism  in  the  administrative  context  that  was 
accepted  by  Council  in  March  2007  reads:  "The  implementation  of  functional 
multilingualism  for  working  and  administrative  purposes  happens  in  a sys- 
tematic  and  goal-oriented  way.  By  means  of  a consultative  process,  strategies 
are  lobbied  and  structures  put  in  place  in  an  ongoing  way  so  as  to  implement 
functional  multilingualism  at  the  NWU  workplace  as  optimally  as  possible, 
with  due  recognition  of  the  language  rights  of  stakeholders".  The  definition 
of  functional  multilingualism  provided  in  the  policy  is: 

"(i)Functional  multilingualism  means  that  the  choice  of  a particular  lan- 
guage  in  a particular  situation  is  determined  by  the  context  in  which  it 
is  used,  and  that  variables  such  as  the  purpose  of  the  communication 
and  levels  of  language  proficiency  of  the  interlocutors  play  a determin- 
ing  role  in  the  choice  of  a particular  language  code  or  language  codes. 
The  implication  is  that  not  all  official  languages  need  to  be  used  for 
communicative  purposes  at  the  NWU  but  that  sensitivity  should  be 
shown  to  the  main  regional  languages  used  in  provinces  where  cam- 
puses  of  the  institution  are  situated. 

(ii)  Multilingual  refers  to  the  use  of  more  than  one  language,  as  well  as  the 
ability  to  use  more  than  one  language." 

The  approved  language  policy  and  plan  of  the  NWU  can  be  accessed  on  the 
institutional  website  at  http://www.nwu.ac.za/language/pdUtaalbeleid_e  .pdf. 
The  quotations  from  the  policy  used  in  this  article  were  accessed  from  the 
website  on  25  October  2007. 

This  article  does  not  focus  on  the  challenges  that  the  NWU  faces  to  estab- 
lish  functional  multilingualism  in  the  educational  and  research  domains  (lan- 
guage  planning  domains  1 and  2)  or  in  the  domain  of  student  life  (language 
planning  domain  3)  or  improvement  of  language  skills  (language  planning 
domain  4).  The  institution  is  already  conducting  groundbreaking  work  in  the 
area  of  language  planning  domain  1,  teaching  and  learning,  with  the  imple- 
mentation  of  simultaneous  interpretation  in  selected  academic  programmes. 
Van  Rooy  (2005)  already  reported  research  findings  of  a project  that  investi- 
gated  the  effectiveness  of  the  simultaneous  interpretation  during  contact  ses- 
sions  at  the  NWU. 

The  focus  of  this  study  is  on  describing  the  challenges  faced  by  administra- 
tors  in  the  institutional  and  campus  offices  of  the  NWU  when  they  attempt  to 
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implement  functional  multilingualism  in  the  language  planning  domains  for 
horizontal  and  vertical  administrative  communication.  This  is  described  in 
language  planning  domain  5 in  the  approved  policy  and  plan.  The  main  aim 
of  the  article  is  to  gather  data  in  a longitudinal  manner  about  the  experiences, 
perceptions  and  attitudes  of  NWU  staff  in  the  process  of  establishing  and  im- 
plementing  a functional  multilingual  language  policy  and  plan  with  regard  to 
administrative  communication.  The  preliminary  results  reported  in  this  arti- 
cle  relate  to  verbal  communication  during  institutional  meetings  at  the  NWU. 
A better  understanding  of  the  experiences,  perceptions  and  attitudes  of  staff 
during  this  process  could  provide  useful  insight  for  language  planners  in  gen- 
eral  and  the  language  pianners  at  the  NWU  in  particular.  A longitudinal  study 
of  the  unique  attempt  by  the  NWU  to  establish  and  implement  a functional 
multilingual  language  policy  in  the  higher  education  context  could  provide 
valuable  data  on  this  part  of  the  transformation  agenda  for  higher  education 
institutions  in  South  Africa. 


2.  Theoretical  framework 

The  research  conducted  in  this  study  is  done  within  the  theoretical  frame- 
work  of  the  ethnography  of  communication  (Saville-Troike,  2003).  The  eth- 
nography  of  communication  is  aimed  at  a study  of  "the  structuring  of  com- 
municative  behaviour  and  its  role  in  the  conduct  of  social  life"  (Saville-Troike, 
2003:  1).  The  ethnography  of  communication  has  two  focal  points:  "the  de- 
scription  and  understanding  of  communicative  behaviour  in  specific  cultural 
settings"  and  the  "formulation  of  concepts  and  theories  upon  which  to  build  a 
global  metatheory  of  hurnan  communication"  (Saville-Troike,  2003: 1-2). 

The  scope  and  focus  of  the  ethnography  of  communication  is  demarcated 
by  the  answer  to  the  following  question:  what  does  a language  user  need  to 
know  to  communicate  appropriately  within  a specific  speech  community?  This 
knowledge  is  defined  as  "communicative  competence"  (Saville-Troike,  2003:  2). 

"The  focus  of  the  ethnography  of  communication  is  the  speech  communi- 
ty,  the  way  communication  within  it  is  patterned  and  organized  as  systems  of 
communicative  events,  and  the  ways  in  which  these  interact  with  all  other 
systems  of  culture"  (Saville-Troike,  2003:  2).  "The  communicative  event  is  the 
basic  unit  for  descriptive  purposes.  A single  event  is  defined  by  a set  of  com- 
ponents  throughout,  beginning  and  involving  the  same  general  purposes  of 
communication,  the  same  general  topic,  and  involving  the  same  participants, 
generally  using  the  same  language  variety,  maintaining  the  same  tone  or  key 
and  the  same  rules  for  interaction  in  the  same  setting"  (Saville-Troike,  2003: 
23).  The  speech  community  in  this  article  is  the  staff  at  the  NWU  and  the 
communicative  events  of  interest  are  institutional  meetings. 
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3.  Methodology 

In  this  study,  it  is  accepted  that,  "Ethnography  involves  an  ongoing  attempt 
to  place  specific  encounters,  events,  and  understandings  into  a fuller,  more 
meaningful  context  ...  ethnography  is  the  continuation  of  fieldwork  rather 
than  a transparent  record  of  past  experiences  in  the  field"  (Tedlock,  2003: 165). 
A key  assumption  in  any  ethnographic  study  is  that  prolonged  interaction 
with  people  will  lead  to  a better  understanding  of  their  beliefs,  motivations, 
and  behaviour  than  by  using  any  other  approach.  In  this  study,  it  is  assumed 
that  prolonged  interaction  with  administrative  staff  at  the  NWU  will  lead  to  a 
better  understanding  of  their  attitudes,  beliefs,  motivations  and  behaviour  in 
terms  of  their  participation  in  administrative  communication  in  an  institution 
where  functional  multilingualism  is  adopted. 

The  main  fieldwork  methods  utilised  by  ethnographers  are  also  employed 
in  this  longitudinal  study:  observations,  interviews  and  an  analysis  of  rele- 
vant  texts  (Campbell  & Gregor,  2003:  71-81;  Miller  et  al,  2003:  219).  In  order  to 
structure  observation  field  notes  effectively,  a grid  is  used  to  capture  field 
notes  during  observation  (see  Appendixes  A,  C and  E as  examples).  These 
grids  each  attempt  to  assist  the  observer  to  capture  both  "holistic"  and  "micro- 
scopic"  data  of  the  relevant  communicative  events  (Miller  et  al,  2003:  224).  No 
attempts  are  made  to  capture  full  communicative  events.  In  line  with  the  fo~ 
cus  of  this  study,  communicative  events  relating  to  the  experiences,  percep- 
tions  and  attitudes  of  administrative  staff  at  the  NWU  who  are  establishing 
and  implementing  a functional  multilingual  language  policy  and  plan  are  cap- 
tured  and  analysed. 

Apart  from  observational  notes,  the  researcher  approaches  individual  mem- 
bers  of  staff  who  voiced  a particular  position  that  reveal  their  experience,  per- 
ception  and  / or  attitude  towards  the  implementation  of  the  functional  multi- 
lingual  language  plan  at  institution  meetings  to  conduct  interviews  with  them. 
The  participants  are  informed  of  the  purpose  of  the  study  and  their  consent  is 
obtained  to  report  the  data  from  the  interviews  anonymously.  The  researcher 
is  in  the  process  of  obtaining  approvaLfrom  the  institutional  ethics  committee 
for  the  longitudinal  part  of  the  study. 

An  important  methodological  issue  is  that  of  participant  observation.  The 
fact  that  the  researcher  is  a participant  in  the  communicative  events  she  is 
observing  should  not  be  glossed  over.  In  line  with  the  ethical  critique  of  eth- 
nographers  about  the  role  played  by  the  participant  observer  (Tedlock,  2003: 
179-180),  the  role,  experiences,  perceptions  and  attitudes  of  the  researcher  in 
the  observation  of  participation  in  communicative  administrative  events  will 
be  noted  and  reported  where  appropriate. 

Interviews  are  conducted  with  relevant  participants  to  (1)  ensure  the  fac- 
tual  correctness  of  the  observer's  field  notes  and  transcripts  related  to  the 
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communicative  event  at  question  and  (2)  to  get  more  information  about  a spe- 
cific  communicative  event.  These  interviews  might  be  formal,  scheduled  events 
or  informal  discussions  with  participants  as  part  of  ordinary  interaction  (Camp- 
bell  & Gregor,  2002:  77).  Interviewees  will  be  identified  as  the  study  progress- 
es  based  on  their  input  at  institutional  meetings  that  relate  to  their  experienc- 
es,  perceptions  and  attitudes  towards  the  implementation  of  the  functional 
multilingual  language  policy  and  plan  of  the  NWU.  An  attempt  will  also  be 
made  to  include  the  voice  of  the  "silent"  participants  in  the  meeting  as  part  of 
the  longitudinal  study. 

Texts  related  to  the  establishment  and  implementation  of  a functional 
multilingual  language  policy  (with  a focus  on  the  administrative  context)  at 
the  NWU  are  identified  and  analysed.  Ultimately,  this  longitudinal  study  hopes 
to  provide  a deeper  insight  into  people's  experiences,  perceptions,  attitudes 
and  understanding  of  a functional  multilingual  language  policy  and  plan  at  a 
university  in  South  Africa. 

The  research  method  applied  in  this  article  included  the  following  "steps". 
The  researcher  attended  institutional  meetings  as  a mernber  and  observed  the 
statements  made  by  colleagues  that  related  to  their  experiences,  perceptions 
and/or  attitudes  towards  the  implementation  of  the  functional  multilingual 
language  policy  and  plan  at  the  institution.  The  observations  were  captured 
in  the  scherne  presented  in  Appendix  A during  the  meeting  or  directly  after 
the  meeting.  After  the  meetings,  the  researcher  approached  the  individual 
and  requested  an  interview  to  talk  about  the  comment/s  made  about  the  im- 
plementation  of  the  functional  multilingual  language  policy  and  plan  by  the 
colleague  during  the  meeting.  The  interviews  were  structured  as  follows:  (a) 
first  of  all,  the  researcher  explained  the  purpose  of  the  research  project;  (b)  the 
permission  of  the  interviewee  was  obtained  to  report  the  data  from  the  inter- 
view  anonymously;  (c)  the  researcher  shared  the  description  of  the  observa- 
tion  of  the  communicative  event  with  the  interviewee  to  check  if  the  observa- 
tions  were  documented  correctly  by  the  researcher;  (d)  the  interviewee  was 
requested  to  explain  her/his  feelings  during  the  time  when  s/he  made  the  state- 
ment  related  to  the  language  policy  and  plan  of  the  NWU;  (e)  specific  ques- 
tions  related  to  the  communicative  event  were  added. 


4.  Presentation  of  preliminary  data 

For  the  purpose  of  this  article,  all  the  data  that  were  obtained  already  as  part 
of  the  study  are  not  reported  in  this  paper.  The  data  reported  in  this  article 
focus  on  an  analysis  of  interviews  with  two  participants  about  three  cornmu- 
nicative  events  that  they  participated  in.  Due  to  the  longitudinal  nature  of  the 
project,  more  data  will  be  added  continuously  until  the  themes  that  emerge 
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from  analyses  of  the  data  start  to  repeat.  In  terms  of  a qualitative  approach 
towards  research,  this  saturation  point  indicates  that  a sufficient  number  of 
communicative  events  have  been  analysed  to  allow  the  publication  of  valid 
and  reliable  results  from  the  project.  No  claims  are  made  about  validity  and 
reliability  of  results  at  this  stage  of  the  project.  The  presentation  of  prelimi- 
nary  data  rather  aims  to  illustrate  the  usefulness  of  this  approach  in  a broad 
attempt  to  gather  data  on  language  experiences,  perceptions  and  attitudes  of 
university  staff  that  are  implementing  a functional  multilingual  language  pol- 
icy  at  the  NWU. 

4. 7 Data  collection 

The  data  discussed  in  this  paper  are  presented  in  Appendixes  A,  C and  E.  This 
data  should  be  seen  as  the  input  material  for  the  interviews  with  the  two  partici- 
pants  whose  experiences,  perceptions  and  attitudes  are  reported  in  this  pa- 
per,  as  explained  above  in  the  methodology  section.  The  input  questions  for 
the  interviews  with  participants  are  presented  in  Appendix  B,  D and  F. 

4.2  Interview  data  with  participant  1 

The  participant  in  Appendix  A and  C is  the  same  person.  Both  communicative 
events  reported  in  Appendix  A and  C were  discussed  with  the  participant  dur- 
ing  the  same  interview.  The  data  related  to  the  second  participant  is  presented 
in  Appendix  E.  The  communicative  events  will  be  discussed  separately. 

4.2. 1 Communicative  event  1 - road  shows  (Appendix  A and  B) 

The  context  of  the  road  shows  is  described  in  detail  in  Appendix  A.  The  main 
elements  to  note  are  that  the  participant  leads  an  institutional  team  in  a road 
show  on  all  campuses  to  explain  the  services,  functions  and  ways  of  work  of 
the  institutional  team.  The  information  was  offered  in  a PowerPoint  presenta- 
tion  and  all  team  members  participated  in  the  presentation.  In  order  to  ac- 
commodate  the  multilingual  nature  of  the  different  campuses,  it  was  decided 
that  participants  would  use  Afrikaans  slides  on  the  Potchefstroom  campus, 
and  the  participant  in  the  research  project  used  Afrikaans  slides,  but  offered 
her  presentation  in  English. 

4. 2.1.1  Assessment  of  correctness  of  description  ofincident  with  the  participant 
An  important  part  of  the  methodology  applied  in  this  project,  was  receiving 
confirmation  from  the  participants  that  the  researcher  documented  the  de- 
tails  of  the  communicative  event  correctly.  The  details,  as  captured  in  Appen- 
dix  A,  were  confirmed  as  correct  by  the  participant  and  the  researcher  ob- 
tained  the  permission  of  the  participant  to  report  the  data  as  part  of  the  re- 
search  project. 
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4.2. 1.2  Description  of  feelings  of  participant  during  the  road  show  presentation 

As  explained  briefly  at  section  4.2.1,  the  participant  wanted  to  accommodate 
the  multilingual  nature  and  policy  of  the  institution  and  therefore  decided  to 
offer  her  presentation  on  the  Potchefstroom  Campus  in  English,  but  to  use 
Afrikaans  slides.  The  input  questions  used  in  the  interview  are  presented  in 
Appendix  B.  The  participant  agreed  that  she  struggled  to  co-ordinate  her  Eng- 
lish  presentation  with  the  Afrikaans  slides.  The  participant  agreed  that  this 
might  have  made  her  seem  "clumsy"  or  "ill-formed"  or  "unconvinced"  about 
the  content  of  her  message.  The  participant  also  agreed  that  this  was  an  unde- 
sirable  situation  for  her  to  be  in,  as  institutional  leader.  Despite  the  difficulties 
experienced,  the  participant  insisted  that  it  was  important  to  use  Afrikaans  at 
the  road  show  session  on  the  Potchefstroom  campus,  because  it  added  "au- 
thority"  to  the  leadership  role  the  participant  plays  institutionally.  It  might 
have  been  better  if  an  Afrikaans  member  of  the  team  operated  the  Afrikaans 
slides  on  behalf  of  the  participant.  It  was  agreed  that  this  would  be  done  in 
similar  situations  in  future.  The  participant  was  upset  by  the  comment  of  the 
school  director  about  the  poor  translation  of  the  term  "Programme  Qualifica- 
tion  Mix".  It  was  translated  as  "programkwalifikasie-mengsel".  The  participant 
felt  that  the  school  director  did  not  appreciate  the  efforts  made  by  the  institu- 
tional  office  to  respect  the  language  preferences  of  the  Potchefstroom  campus. 

4.2.2  Communicative  event  2 - strategic  planning  session  by  extended 
senior  management  (January  2006)  (Appendixes  C and  D) 

The  full  details  of  the  context  of  this  communicative  event  are  presented  in 
Appendix  C.  The  input  questions  for  the  interview  with  the  participant  are 
presented  in  Appendix  D.  The  communicative  event  took  place  at  a senior 
management  planning  meeting  at  the  beginning  of  2006.  During  the  meeting, 
a lot  of  Afrikaans  was  used  by  colleagues.  At  the  end  of  the  meeting  the  chair- 
person  requested  feedback  on  the  planning  session  from  all  participants.  The 
participant  was  the  third  last  person  to  comment  and  in  her  comments  she 
requested  colleagues  who  use  Afrikaans  at  these  meetings  to  remember  that 
there  are  second  and  third  language  users  of  Afrikaans  at  the  meeting  and 
that  they  should  adjust  the  pace  and  choice  of  vocabulary  to  enable  those 
colleagues  to  also  follow  their  comments.  A point  of  interest  was  that  inter- 
pretation  services  were  available  at  the  meeting,  but  that  the  participant  chose 
not  to  make  use  of  the  service. 

4.2.2. 1 Assessment  of  correctness  of  description  of  communicative  event  with  the 
participant 

Again,  the  methodology  applied  in  the  project  required  of  the  researcher  to 
confirm  the  correctness  of  the  description  of  the  communicative  event  (full 
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details  in  Appendix  C)  with  the  participant.  In  this  case,  the  participant  con- 
firmed  that  the  communicative  event  was  represented  correctly  in  the  field 
notes  of  the  observer  and  she  did  not  want  to  add  or  delete  sections  of  the 
description  of  the  event  in  the  field  notes. 

4. 2. 2. 2 Description  of  feelings  of  participant  when  utterance  was  made 

The  main  emotion  that  underpinned  the  utterance  made  by  the  participant 
that  constituted  the  communicative  event  relevant  to  this  paper,  was  that  of 
anger.  The  participant  argued  as  follows:  "My  English  proficiency  is  much 
better  than  that  of  the  colleagues  around  me.  But  I am  not  arrogant  about  it.  I 
do  not  use  big  words  or  speak  fast  when  I speak  English  at  work,  because  I 
know  that  some  colleagues  might  not  follow  me".  The  participant's  feeling 
was  that  she  took  great  care  when  she  used  English  to  speak  slowly  and  to 
select  words  that  less  proficient  English  speakers  would  understand.  She  did 
not  experience  the  same  care  when  Afrikaans  speaking  people  used  Afrikaans 
at  institutional  meetings.  This  indicated  to  her  that  these  speakers  did  not 
want  to  accommodate  or  reach  out  to  her,  and  that  made  her  feel  angry. 

Furthermore,  she  argued  that  she  went  through  the  pain  of  using  English, 
although  it  was  not  her  mother  tongue,  to  accommodate  colleagues  who  did 
not  understand  her  mother  tongue.  She  felt  great  pity  for  people  who  had  to 
use  English  while  they  were  not  proficient  enough  in  English.  But  she  experi- 
enced  mother  tongue  speakers  of  Afrikaans  who  used  Afrikaans  at  a fast  pace 
and  who  did  not  carefully  select  their  words  to  improve  the  chances  of  less 
proficient  Afrikaans  speaking  colleagues  to  understand  them,  as  arrogant. 

4. 2. 2. 3 Use  of  interpretation  services  by  the  participant 

In  response  to  the  question  related  to  the  use  of  the  interpretation  services, 
the  participant  maintained  that  she  understood  Afrikaans  and  therefore  she 
did  not  need  the  interpretation  services.  She  offered  Afrikaans  as  a subject  in 
her  undergraduate  and  honours  degrees  and  she  stated  that  her  reading  and 
writing  proficiency  in  Afrikaans  were  good.  She  agreed  that  her  speaking  and 
listening  proficiency  in  Afrikaans  were  not  good,  because  she  did  not  have 
opportunities  to  practice  it  for  many  years„ 

She  believed  that  her  ability  to  understand  Afrikaans  contributed  substan- 
tially  to  her  acceptance  by  some  colleagues  at  the  institution.  She  maintained 
that  her  use  of  the  interpretation  services  was  unnecessary  if  colleagues  that 
used  Afrikaans  took  care  when  they  used  it  to  speak  slowly  and  to  select  less 
"technical"  or  "academic"  words.  She  argued  that  in  our  multilingual  work- 
place,  we  should  all  be  compromising  to  accommodate  each  other.  She  com- 
promised  by  using  English  and  by  speaking  slowly  and  selecting  her  words 
carefully.  When  Afrikaans  speaking  colleagues  did  not  do  the  same,  she 
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"switched  off"  and  felt  that  she  was  not  part  of  the  conversation  and  deci- 
sions  that  followed  any  more. 

4.2.3  Communicative  event  3 - Strategic  planning  session  by  extended  senior 
management  (November  2005)  (Appendixes  E and  F) 

The  details  of  the  communicative  event  are  presented  in  Appendix  E and  the 
input  questions  for  the  interview  with  the  participant  are  presented  in  Ap- 
pendix  E This  was  a two  day  planning  meeting  of  the  senior  management 
team  of  the  NWU  that  was  held  away  from  the  campuses.  Interpretation  serv- 
ices  were  available  and  full  group  discussions  as  well  as  break  away  sessions 
into  smaller  groups  took  place.  At  the  end  of  the  planning  meeting,  the  chair- 
person  requested  participants  to  share  their  views  about  the  effectiveness  of 
the  session  with  the  group.  The  participant  raised  the  issue  of  the  use  of  Afri- 
kaans  at  the  rneeting  as  a problem.  He  communicated  that  he  could  not  un- 
derstand  why  colleagues  who  are  perfectly  conversant  in  English  continued 
to  use  Afrikaans  at  these  meetings.  He  communicated  frustration  and  anger  at 
people  who  assumed  that  every  person  understood  Afrikaans  and  he  was 
particularly  upset  at  the  use  of  Afrikaans  in  the  breakaway  groups  where  the 
interpretation  services  were  not  available. 

4.2.3. 1 Assessment  of  correctness  of  description  of  communicative  event  with  the 
participant 

The  participant  agreed  with  the  correctness  of  the  description  of  the  commu- 
nicative  event  at  the  first  interview  with  the  researcher.  After  the  interview, 
the  researcher  sent  the  notes  to  the  participant  and  the  participant  had  an 
opportunity  to  add  to  the  description. 

4. 2. 3. 2 What  did  you  feel  when  you  raised  the  issue? 

The  participant  expressed  frustration.  The  participant  regarded  English  as  the 
"operational  language"  in  South  Africa.  I think  a definition  of  "operational 
language"  would  be  the  language  which  all  South  Africans  understand.  I will 
need  to  clarify  this  definition  with  the  participant  in  future.  The  participant 
was  frustrated  by  the  knowledge  that  many  of  the  people  who  used  Afrikaans 
in  sessions  were  able  to  speak  English.  It  did  not  make  sense  to  the  participant 
that  they  continued  to  use  Afrikaans. 

The  participant  was  aware  that  some  participants  claimed  that  they  strug- 
gled  to  express  themselves  fully/with  ease  in  English.  The  participant  did  not 
believe  this  claim.  The  participant  said  that  many  of  them  used  English  very 
well.  It  was  no  problem  to  the  participant  that  colleagues  used  an  Afrikaans 
word  when  they  struggled  to  find  the  right  English  word.  Another  colleague 
usually  helped  the  person  out  with  the  correct  English  word  and  the  partici- 
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pantbelieved  that  this  practice  was  good  for  language  learning.  The  colleague 
who  struggled  to  find  the  word  could  learn  a new  English  word  and  other 
colleagues  could  learn  an  Afrikaans  word. 

Another  line  of  thinking  by  the  participant  was  that  management  should 
stop  to  waste  so  much  money  and  resources  to  implement  a functional  multi- 
lingual  language  policy.  In  the  opinion  of  the  participant,  management  should 
decide  that  English  is  the  operational  working  language  for  the  institution. 
The  participant  observed  that  management  had  taken  other  "unpopular"  de- 
cisions  that  did  not  necessarily  reflect  the  view  of  all  employees  (e.g.  about 
institutional  credit  weightings  for  modules),  and  that  was  fine,  because  in  the 
end  management  had  the  mandate  to  take  these  decisions.  The  participant 
believed  the  same  should  be  done  with  a management  declaration  that  Eng- 
lish  was  the  operational  working  language  of  the  institution. 

The  participant  was  frustrated  by  the  institution's  waste  of  money  regard- 
ing  its  insistence  on  using  the  interpretation  services  at  meetings  (e.g.  the  sen- 
ior  management  meeting  and  the  Institutional  Senate  meetings).  The  partici- 
pant  said  that  very  few  people  used  the  earphones  when  colleagues  used 
English  and  the  participant  was  one  of  about  3 people  in  these  meetings  who 
used  the  earphones  when  Afrikaans  was  used.  The  participant  asked  me  if  I 
knew  how  many  people  could  speak  ONLY  Afrikaans  at  the  institution,  be- 
cause  the  interpretation  service  would  be  imperative  for  them. 

The  participant  was  also  frustrated  by  the  "disruptive"  effect  that  the  inter- 
pretation  service  had  for  him  at  meetings.  The  participant  was  particularly 
frustrated  when  the  interpreters  were  too  slow  (in  his  opinion)  and  the  partici- 
pant  had  the  experience  that  his  concentration  span  was  broken  when  he  had 
to  put  on  the  earphones,  and  had  to  take  them  off  again  and  vice  versa  through- 
out  the  meeting. 

4. 2. 3. 3 Why  does  it  bother  you  that  colleagues  use  Afríkaans?  The  interpretation 
services  are  supposed  to  ensure  that  we  understand  what  is  said  at  the 
meeting.  What  is  your  opinion  about  the  use  of  the  interpretation  services? 
The  participant  shared  an  important  part  of  his  personal  history  to  answer 
this  question.  Due  to  apartheid  laws,  the  family  of  the  participant  relocated  to 
England  when  he  was  about  3 or  4 years  old.  The  p^rticipant's  parents  got 
divorced  in  England  and  the  participant's  father  died  before  he  was  8 years 
old.  The  participant  stated  that  he  did  not  have  a pleasant  childhood.  These 
experiences  came  as  a direct  result  of  the  apartheid  policies  in  South  Africa 
and  Afrikaans  was  directly  related  to  that. 

On  the  other  hand,  the  participant  stated  that  this  experience  also  had 
positive  results.  It  took  him  to  other  countries  where  he  had  opportunities  to 
learn  and  study.  The  participant  might  not  have  had  the  same  opportunities  if 
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the  family  stayed  in  South  Africa.  The  participant  tried  to  see  the  positive  side 
to  the  experience  as  well.  However,  the  participant  had  mixed  feelings  when 
he  heard  or  thought  about  Afrikaans. 

The  participant  did  not  see  the  interpretation  services  as  a tool  that  assist- 
ed  in  facilitating  an  effective  work  environment  in  a multilingual  context.  The 
participant  was  frustrated  by  the  "disruptive"  element  of  the  interpretation 
services  on  his  ability  to  participate  in  meetings  and  the  expense  of  the  inter- 
pretation  services  bothered  him  as  well.  According  to  the  participant  the  in- 
terpretation  services  did  not  add  anything  to  the  effectiveness  of  the  meeting. 
The  participant's  opinion  remained  that  colleagues  could  all  use  English,  and 
that  meetings  would  be  more  effective  if  all  colleagues  used  English. 

The  participant  also  expressed  that  he  did  not  believe  it  was  sensible  for 
colleagues  to  use  any  language  other  than  English  at  the  institution.  Although 
he  was  able  to  speak  SeSotho,  he  claimed  that  he  resisted  using  it  in  class  or 
meetings,  because  he  deemed  it  an  "unfit"  thing  to  do  at  the  institution.  I am 
not  sure  what  languages  the  participant  can  speak.  It  might  be  that  time  in 
exile  resulted  in  him  being  able  to  speak  English  well,  but  had  the  detrimental 
effect  of  providing  no  opportunities  to  gain  proficiency  in  other  South  African 
languages.  If  this  is  true,  then  any  institutional  attempt  at  multilingualism 
will  disadvantage  and  "threaten"  this  participant.  The  researcher  will  have  to 
explore  this  issue  further  with  the  participant. 


5.  Discussion  and  interpretation  of  data 

As  stated  earlier  in  the  article  the  data  presented  in  this  paper  should  be  re- 
garded  as  preliminary  data,  due  to  the  qualitative  and  longitudinal  approach 
adopted  for  this  study.  No  claims  are  made  that  the  data  presented  in  this 
paper  would  provide  finite  conclusions  about  this  matter.  The  discussion  and 
interpretation  of  data  from  the  article  is  therefore  brief  and  tentative. 

5.  / Brief  discussion  and  tentative  interpretation  of  current  data,  as  well  as 
tentative  implications  of  findings 

In  terms  of  the  data  reported  in  this  paper  (presented  in  Appendixes  A - F),  at 
least  three  issues  should  be  discussed.  The  first  issue  relates  to  the  notion  of 
"power"  and  language  use.  In  both  the  communicative  events  related  in  Ap- 
pendixes  A and  C,  the  participant  demonstrates  that  she  is  aware  of  issues 
related  to  language  and  power.  In  both  instances,  the  participant  expressed 
notions  such  as  added  "authority"  to  her  institutional  leadership  and  the  no- 
tion  of  "investment"  when  she  uses  Afrikaans  in  meetings  on  the  Potchef- 
stroom  campus  and  institutional  meetings. 
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In  contrast  to  this,  the  participant  in  Appendix  E challenges  the  usefulness 
of  Afrikaans  in  the  institution.  The  painful  personal  experiences  of  the  partic- 
ipant  make  it  impossible  for  him  to  see  any  useful  role  for  Afrikaans  at  the 
institution.  This  could  be  regarded  as  one  of  the  mainstream  views  about  the 
use  of  Afrikaans  in  post-1994  South  Africa,  and  although  applied  linguists  and 
language  planners  might  view  this  as  an  unproductive  manner  in  which  to 
view  the  role  of  Afrikaans  in  the  post-1994  South  Africa,  the  views  offered  by 
this  participant  certainly  could  not  be  ignored  by  the  NWU.  Furthermore,  it 
might  be  that  the  multilingual  proficiency  of  this  participant  is  lower  than 
that  of  many  of  the  colleagues  around  him  and  that  the  notion  of  "multilin- 
gualism"  might  seem  "threatening"  to  him? 

What  are  possible  implications  from  these  tentative  findings?  The  manag- 
ers  of  the  language  policy  and  plans  of  the  NWU  should  take  note  of  these 
attitudes  and  realise  that  even  more  communication  about  the  benefits  of  bi- 
lingualism  should  be  distributed  in  the  institution.  Furthermore,  the  institu- 
tion  needs  to  assure  colleagues  who  are  not  multilingual  that  they  will  not  be 
excluded  from  participating  in  the  operations  of  the  institution.  A possible 
perception  of  any  "hegemony"  of  multilingualism  might  be  detrimental  in 
the  institution's  attempt  to  include  all  colleagues. 

Secondly,  the  perceived  "language  arrogance"  of  Afrikaans  speakers  using 
"technical"  or  "academic"  words  at  a fast  pace  at  meetings  is  an  important 
notion  to  consider.  The  Afrikaans  users  at  this  specific  meeting  are  probably 
not  aware  of  the  very  explicit  meaning  this  participant  attaches  to  their  use  of 
"technical"  or  "academic"  Afrikaans  at  a fast  pace.  They  unconsciously  proba- 
bly  also  assume  that  due  to  the  availability  of  interpretation  services,  they  do 
not  need  to  adjust  their  language  use  at  all,  because  non-Afrikaans  speaking 
colleagues  would  receive  appropriate  interpretation  in  English. 

It  seems  as  if  these  assumptions  and  the  consequent  arrangements  made 
in  terms  of  interpretation  services  belie  the  complexity  of  the  levels  of  multi- 
lingualism  of  colleagues  at  the  institution.  An  underlying  assumption/percep- 
tion  is  that  colleagues  either  understand  Afrikaans  fully,  or  do  not  understand 
it  at  all,  therefore  providing  interpretation  services  is  sufficient  in  implement- 
ing  the  functional  multilingual  language  policy  of  the  institution. 

An  analysis  of  this  communicative  event  indicates  that  this  arrangement  is 
not  enough  to  facilitate  participation  at  this  multilingual  workplace.  This  analy- 
sis  seems  to  suggest  that  more  should  be  done  to  sensitise  and  train  colleagues 
at  the  institution  about  language  habits  and  behaviour  that  would  facilitate 
optimal  participation  of  all  members  at  meetings.  One  important  notion  that 
colleagues  need  to  be  aware  of,  is  that  individuals  at  these  meetings  display 
different  levels  of  proficiency  in  terms  of  different  languages.  This  is  a given 
in  any  multilingual  workplace.  Apart  from  arranging  for  interpretation  serv- 
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ices  at  these  multilingual  institutional  meetings,  all  colleagues  attending  these 
meetings  should  be  sensitised  and  trained  that  we  should  all  use  a slower 
pace  and  select  our  words  more  carefully  to  enhance  the  opportunities  of  sec- 
ond  and  third  language  users  to  participate  optimally  at  these  meetings.  Fur- 
thermore,  this  very  reasonable  request  might  enhance  the  ability  of  the  inter- 
preters  to  conduct  their  jobs  at  an  even  higher  level  of  excellence  as  well.  These 
tentative  results  from  this  singular  analysis  of  a communicative  event  at  a 
multilingual  workplace  seem  to  suggest  that  there  are  training  and  develop- 
ment  needs  that  should  form  part  of  the  implementation  plan  for  a functional 
multilingual  language  policy  and  plan  at  this  institution.  All  colleagues  should 
be  sensitised  and  trained  in  terms  of  the  pace  and  selection  of  words  when 
they  participate  at  these  meetings.  Furthermore,  we  need  to  spend  more  time 
to  ensure  that  we  provide  our  contributions  in  such  a manner  that  the  inter- 
preters,  as  well  as  second  and  third  language  users  of  languages  used  at  the 
meeting,  have  the  best  possible  chance  to  participate  optimally  at  the  meet- 
ing. 

6.  Conclusions 

One  of  the  tentative  conclusions  from  this  case  study  is  that  the  complexities 
of  the  levels  of  multilingual  language  proficiency  need  to  be  recognised  to 
design  appropriate  mechanisms  and  support  structures  to  enable  optimal  par- 
ticipation  of  all  colleagues  in  multilingual  meetings.  It  seems  that  the  use  of 
interpretation  services  as  the  only  strategy  to  facilitate  participation  in  multi- 
lingual  meetings,  without  considering  the  levels  of  different  multilingual  pro- 
ficiencies  brought  to  the  discussion  table  in  multilingual  settings,  could  have 
unintended  and  opposite  effects  on  the  levels  of  participation  of  colleagues. 
An  acknowledgement  of  the  different  levels  of  language  proficiency  of  multi- 
lingual  colleagues  raises  the  level  of  sophistication  a multilingual  institution 
should  invest  in  if  they  are  serious  about  implementing  a functional  multilin- 
gual  language  policy  and  plan.  The  need  for  sensitisation  and  training  and 
development  opportunities  for  colleagues  participating  in  multilingual  meet- 
ings  seems  imperative  following  on  the  analysis  from  this  one  case  study. 

A second  tentative  conclusion  is  that  a qualitative,  longitudinal  approach 
seems  to  be  very  productive  to  reveal  the  intensely  personal  and  individual 
attitudes,  experiences  and  perceptions  of  language  users  that  influence  their 
language  behaviour,  as  demonstrated  by  communicative  events  in  which  they 
participate.  Along  with  other  methods  and  approaches,  data  gathered  in  this 
manner  might  provide  language  planners  and  managers  at  universities  in 
South  Africa  with  relevant  conclusions  upon  which  decisions  for  language 
planning  could  be  based. 
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Ultimately,  insights  from  this  study  that  aims  to  gather  data  in  a longitudi- 
nal  manner  could  be  used  by  language  planners  and  language  planning  theo- 
rists  to  adjust  existing  assumptions  about  the  pros  and  the  cons  of  language 
planning  in  multilingual  contexts.  Eventually,  this  data  would  provide  insight 
into  the  effect  of  implementing  a functional  multilingual  language  policy  and 
plan  on  the  nature  and  level  of  participation  in  this  university  as  a workplace. 
This  brave  attempt  by  the  NWU  to  implement  a functional  multilingual  lan- 
guage  policy  and  plan  deserves  the  careful  attention  of  a longitudinal  study 
of  this  nature. 
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APPENDIX  A - Road  shows  by  the  NWU 


Type  of  event: 

Date: 

Venues: 

Nr  of  people: 

Road  shows  by  division  of  an  institutional  manger  at  the  NWU 

7, 10  and  17  March,  2005 

Senate  Halls  and  Lecture  Hall 

About  50  on  Potchefstroom  Campus; 

about  25  on  Mafikeng  and  Vaal  Triangle  Campuses 

Role/relationship 
of  observer  in  event: 

Member  of  Extended  Senior  Management  team; 
participant  in  the  presentation 

Category 

Field  notes 

Scene 

Type  of  event:  A meeting  (information  session  & discussion) 

Topic:  Information  about  functions  of  institutional  office 

Purpose:  To  inform  academic  managers  about  the  following: 

i.  the  gist  of  functions  that  will  be  performed  by  this  institutional  office 

ii.  the  division  of  functions  related  to  this  institutional  office,  that  is  between 
campus  and  institutional  levels 

iii.  clarifying  the  communication  channels  between  the  officers  in  the 
institutional  office  and  campuses  related  to  this  portfolio. 

Setting 

Potchefstroom  campus  (7  March  2005,  09:00-11:00): 

The  meeting  was  held  in  the  Joon  van  Rooy  Senate  Hall  which  is  a formal  lecture 
room  with  fixed  seats,  a fixed  front  desk  and  fixed  data  projection  facilities. 

Vaal  Triangle  campus  (10  March  2005,  10:00-12:00): 

The  meeting  was  held  in  the  Old  Mutual  lecture  room  which  is  a venue  with  fixed 
seats,  a fixed  front  desk  and  fixed  data  projection  facilities. 

Mafikeng  campus  (17  March  2005,  10:00-12:00): 

The  meeting  was  held  in  the  Senate  Room  which  is  a venue  with  fixed  tables  and 
seats,  arranged  in  the  form  of  an  oval,  with  fixed  data  projection  facilities. 

Emotional  tone 

of  the  event 

Potchefstroom  campus:  It  was  a serious  meeting,  with  some  hostile  and  some 
supportive  elements.  About  50  academic  managers  (white,  majority  male)  attended 
the  meeting.  The  campus  rector  (white  female)  was  present.  There  was  engagement 
with  the  ideas  in  the  presentation.  A discussion  of  about  30  minutes  was  conducted 
after  the  presentation.  A dean  (white  male)  was  unhappy  after  the  presentation  about 
the  possible  extra  cost  for  the  numerous  staff  that  needs  to  be  appointed  in  the 
institutional  office,  and  there  was  a remark  by  a school  director  (white  male)  about  a 
term  on  the  Afrikaans  slides  of  the  IM  member.  The  school  director  felt  the  translation 
was  poor.  The  terms  in  question  were:  “Programme  Qualification  Mix"  that  was 
translated  as  “Programkwalifikasiemengser.  The  campus  rector  joked  and  replied 
that  in  future  we  will  continue  to  use  “Programme  Qualification  Mix”.  Two  colleagues 
(white  females)  were  supportive  about  an  increase  in  staff  in  the  institutional  office. 
They  argued  that  we  need  to  expand  the  staff  complement  even  further,  because  the 
work  planned  by  this  institutional  division  was  important  and  the  workload  was  too 
heavy  for  the  staff  suggested. 

Vaal  Triangle  campus:  It  was  a serious  meeting,  attended  by  about  25  people 
(white,  majority  male),  with  no  discussion  after  the  presentation.  After  we  prompted 
some  discussion,  the  campus  rector  replied  that  the  ideas  presented  will  be 
discussed  on  relevant  internal  campus  forums.  One  colleague  did  offer  some  input: 
he  raised  the  idea  of  institutional  budgets  (as  opposed  to  campus  budgets)  for  some 
functions. 
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Participants 


Message  form 


Message  content 


Perception  of  the 
effectiveness  of  the 
communicative  event 


Mafikeng  campus:  It  was  a serious  meeting,  attended  by  about  25  people  (black, 
majority  male),  with  a fair  amount  of  discussion  after  the  presentation.  The  tone  of 
the  discussion  was  constructive  and  critical.  It  was  constructive  in  the  sense  that 
campus  colleagues  agreed  with  the  general  gist  of  the  presentation,  but  it  was  also 
critical,  because  colleagues  requested  some  information  about  service  level 
agreements.  They  were  concerned  that  service  delivery  of  functions  performed  by  an 
institutional  office  on  campuses  might  be  delayed  and  they  requested  information 
about  the  mechanisms  and  procedures  the  institutional  office  plans  to  put  in  place  to 
minimize  this  concern. 


The  staff  from  the  institutional  office  included  an  IM  member  (a  black  woman)  and 
five  directors  (1  black  woman,  1 white  woman  and  three  white  men)  that  operate  on 
institutional  level  in  the  portfolio  of  the  IM  member.The  participants  in  the  audiences 
were  the  academic  managers  on  all  campuses.  The  Campus  Rector,  Vice-Rector: 
Academic,  Deans,  School  Directors  / Heads  of  Department,  Campus  Registrars  and 
relevant  campus  academic  support  service  staff  were  invited.  On  the  Mafikeng 
campus  the  majority  of  the  audience  was  comprised  of  black  men;  on  the 
Potchefstroom  campuses  the  majority  of  the  audience  was  comprised  of  white, 
Afrikaans-speaking  men  and  women;  and  on  the  Vaal  Triangle  campus,  the  majority 
of  the  audience  was  comprised  of  white,  Afrikaans-speaking  men. 


Spoken  South  African  English,  supported  by  PowerPoint  slides,  prepared  for  each 
campus.  The  main  decision  about  message  form  was  which  language/s  should  be 
used  for  the  verbal  presentations  and  on  the  PowerPoint  slides?  The  institutional 
office  decided  to  develop  English  PowerPoint  slides,  with  an  Afrikaans  translation  of 
the  IM  member’s  section  to  be  used  on  the  Potchefstroom  campus.  It  was  decided 
that  the  IM  member  will  use  Afrikaans  slides  at  Potchefstroom,  although  she  will 
present  in  English,  and  that  the  rest  of  the  division  will  use  English  slides  at 
Potchefstroom,  but  that  we  will  present  in  Afrikaans.  On  the  other  two  campuses 
(Mafikeng  and  Vaal  Triangle),  we  all  used  the  English  slides,  and  we  presented  in 
English  as  well.  There  is  no  budget  or  other  internal  support  for  the  translation  of 
documents  used  by  the  institutional  office.  Therefore,  I offered  to  do  the  translation  of 
the  IM  member's  slides  into  Afrikaans.  I had  very  little  time  to  do  this,  and  eventually  I 
ended  up  doing  it  during  an  institutional  senate  meeting.  I knew  the  translation  was 
not  perfect,  and  I struggled  to  find  Afrikaans  terms  for  national  bodies  such  as  the 
HEQC  and  terms  used  by  the  Department  of  Education,  like  PQM.  I was  seated  next 
to  the  campus  rector  of  the  Potchefstroom  campus  and  she  assisted  me  with  the 
translation  of  some  terms  during  the  senate  meeting. 


A copy  of  the  PowerPoint  presentation  slides  is  available  for  interested  colleagues. 

A summary  of  the  message  content  would  be: 

• Members  from  the  institutional  office  wish  to  inform  campus  academic  managers 
of  their  vision  of  the  work  that  must  be  conducted  by  this  particular  institutional 
office;  and 

• Campus  academic  managers  should  respond  critically  to  these  suggestions  by 
providing  input  that  would  tailor-make  the  functions  performed  by  the  institutional 
office  for  their  campus. 


Potchefstroom  campus:  From  the  perspective  of  the  IM  office,  the  communicative 
event  was  successful.  The  audience  interacted  with  the  institutional  office  in  the 
discussion  about  the  proposal  and  there  was  very  little  disagreement  with  the 
proposal.  The  negative  issues  that  were  raised  (e.g.  increased  expenses  because  of 
huge  institutional  staff  complement)  could  be  addressed  (there  are  no  new  positions 
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created  for  the  institutional  office,  existing  positions  from  the  campuses  are  migrated 
to  the  institutional  office),  and  the  positive  suggestions  / input  was  added  to  the  final 
proposal.lf  I consider  the  effectiveness  of  the  decisions  that  were  taken  about 
message  form  (translation  of  the  IM  member’s  slides  into  Afrikaans,  while  she 
presented  in  English;  and  the  English  slides  of  the  other  presenters,  used  while  they 
presented  in  Afrikaans),  I don't  think  the  communicative  event  was  wholly  successful. 
The  IM  member  is  a seasoned  public  presenter  that  uses  PowerPoint  well.  She  was 
very  uncomfortable  with  the  Afrikaans  slides  while  she  presented  at  the 
Potchefstroom  campus.  She  struggled  to  synchronize  her  English  verbal 
presentation  with  herAfrikaans  slides.  This  made  her  seem  “clumsy”  or  “ill-formed”  or 
"unconvinced”  about  the  content  of  her  message.  This  is  a dangerous  situation  for 
her  to  be  in,  as  institutional  leader.  She  is  the  first  black  female  institutional  leader 
that  has  to  interact  with  the  dominantly  white,  Afrikaans,  male  academic  community 
on  the  Potchefstroom  campus.  Particularly  at  the  beginning  of  the  merging  process, 
she  cannot  afford  to  appear“clumsy”  or  "ill-informed”,  because  these  members  might 
decide  that  she  is  "incompetenf  and  therefore  choose  to  ignore  her  or  patronize  her 
in  future.  The  negative  remark  from  a director  about  a term  that  was  translated 
“incorrectly”  according  to  him,  also  puts  a question  mark  on  the  possible  success  of 
the  decision  to  translate  the  IM  member’s  slides  into  Afrikaans  forthe  Potchefstroom 
presentation. 

Vaal  Triangle  campus:  I am  not  sure  that  the  communicative  event  was  successful 
from  the  perspective  of  the  institutional  office.  The  audience  did  not  engage  with  the 
proposal,  because  there  was  no  immediate  discussion  after  the  presentation.  The 
single  request  from  a campus  manager  (for  institutional  budgets  for  certain  activities) 
was  added  to  the  final  proposal.  However,  in  follow-up  meetings  with  the  campus 
rector,  dean  and  chief  director  for  infrastructure  and  facilities,  it  became  clear  that  the 
message  was  internalised,  that  it  was  discussed  at  other  internal  forums  at  that 
campus,  and  it  resulted  in  fairly  serious  human  resources  activities  to  restructure 
some  campus  staff  so  that  campus  functions  proposed  by  the  IM  member  and 
institutional  team  could  be  effected.  The  communicative  event  was  therefore  very 
successful.  If  I consider  the  effectiveness  of  the  decisions  that  were  taken  about 
message  form  (using  English  slides  and  presenting  in  English),  I think  the 
communicative  event  was  fairly  successful.  It  seems  that  colleagues  had  to 
intemalize  the  message  first,  and  that  they  could  not  react  to  it  immediately. 

When  they  did  react  later  (after  internal  discussions),  it  reflected  their  deep 
understanding  of  the  message.  The  influence  of  using  only  English  (on  the  slides  and 
in  the  presentation)  should  be  investigated  in  future  as  an  explanation  for  the  lack  of 
initial,  immediate  response  to  the  presentation  on  this  campus. 

MafiKeng  campus:  I am  not  convinced  that  the  communicative  event  was  successful 
from  the  perspective  of  the  institutional  office.  The  audience  engaged  with  the 
proposal  in  a critical,  but  constructive  manner  that  seemed  to  indicate  that  they 
understand  the  proposal  and  that  they  are  willing  to  assist  with  its  implementation, 
provided  that  service  level  agreements  are  stipulated.  However,  when  I follow-up 
discussions  with  academic  managers  on  the  Mafikeng  campus,  it  was  clear  that  they 
did  not  understand  all  the  nuances  in  the  proposal.  I had  to  spend  quite  a lot  of  time 
to  make  sure  that  all  the  details  of  the  proposal  are  understood.  If  I consider  the 
effectiveness  of  the  decision  that  was  taken  about  message  form  (using  English 
slides  and  presenting  in  English),  I am  not  sure  what  the  possible  effect  of  using 
English  is  on  the  delay  in  information  processing  that  I perceived  in  the  follow-up 
discussions  with  academic  managers.  The  influence  of  using  only  English  (on  the 
slides  and  in  the  presentation)  should  be  investigated  in  future  as  an  explanation  for 
seeming  delay  in  processing  of  the  details  of  the  message  on  this  campus. 
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What  is  learned  about 

functional 

multilinguaiism? 

a)  Revisit  the  idea  to  put  the  IM  member  in  the  difficult  position  to  use  Afrikaans 
slides  while  she  presents  in  English.  It  might  make  her  seem  “clumsy"  and  this  may 
damage  the  impact  of  the  message  she  brings,  particularly  because  she  is  one  of  the 
first  black  female  managers  that  addresses  the  mainly  white,  Afrikaans  academics  on 
the  Potchefstroom  campus. 

b)  Don't  try  to  accommodate  the  Afrikaans  audience  by  translating  your  slides  into 
Afrikaans  yourself.  They  do  not  appreciate  the  effort,  they  are  critical  of  your  attempts 
without  offering  better  translations.  It  might  be  better  to  apologise  for  the  fact  that  you 
conduct  your  presentation  in  English  at  the  beginning,  and  to  continue  in  English. 

c)  Or,  as  an  alternative  to  (b),  find  appropriate  mechanisms  and  processes  to  inform 
Afrikaans-speakers  of  the  fact  that  the  institutional  office  would  like  to  provide 
documentation  in  Afrikaans  and  English,  but  that  there  is  no  budget  or  other  support 
for  this  decision,  and  therefore,  the  audience  should  please  be  tolerant  with  possible 
errors  that  might  occur  and  the  audience  should  know  that  the  institutional  office 
would  appreciate  constructive  solutions  to  possible  inappropriate  translations. 

Other  comments 

None 

APPENDIX  B - 

Specific  communicative  event  St  participant  information 

Date: 

Venue: 

Nr  of  people: 
Participant  1: 

7 March,  2005 

Senate  Hall  (Potchefstroom  campus) 

About  50 

Institutional  Management  (IM)  member 

Gender:  Female;  Race:  African;  Age:  49 

Partícipant  2: 

Gender:  n/a;  Race:  n/a;  Age:  n/a 

Relatíonship 
between  partícipants 

& observer:  Observer  reports  to  IM  member 


Participant 

Information  about  dialogue  / communicative  event 

Field  notes  about  dialogue 

Observer 

No  specific  dialogue  relevantto  the  communicative  event.  The  issue  the  observer 
discussed  with  the  participant  related  specifically  to  the  road  show  conducted  on  the 
Potchefstroom  campus:  how  effective  was  the  choice  of  the  team  to  provide 
PowerPoint  slides  in  Afrikaans  for  the  IM  member,  while  she  used  English  to  present? 
The  field  notes  in  Appendix  A that  present  a reflection  on  the  effectiveness  of  the 
communication  at  the  event  provides  the  input  for  the  interview  with  participant  1. 

Other  comments  / 
field  notes  / questions 
to  ask  participant 
in  interview 

a)  Ask  participant  to  relate  incident  first.  Then  check  if  my  field  notes  are  correct.  I.e. 
observer  experienced  that  participant  struggled  to  present  her  presentation  from  the 
Afrikaans  slides?  Is  this  true? 

b)  Explain  your  feelings  when  you  made  the  utterance  / during  the  presentation? 
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APPENDIX  C - Planning  meeting  of  Senior  Management  of  NWU 


Type  of  event: 

Date: 

Venues: 

Nr  of  people: 
Role/relationship 
of  observer  in  event: 

Formal  planning  meeting  of  Extended  Senior  Management  of  NWU 
23  January,  2006 

Transnet  Room  (Potchefstroom) 

About45 

Member  of  Extended  Senior  Management  team;  attending  with  line 
manager  & other  peers  in  portfolio  of  line  manager 

Category 

Field  notes 

Scene 

Type  of  event:  Formal  meeting  called  by  Vice-Chancellor 

Topic:  Campus  feedback  on  Institutional  Office  & Discussion  of  IP 

Purpose:  (a)  Allow  campuses  an  opportunity  to  discuss  frustrations/concerns  about 
services  delivered  by  Institutional  Office 
(b)  Discussion  of  IP  by  Institutional  Office  staff 

Setting:  Formal  conference  venue,  chairs  organised  in  horse  shoe  shape,  no 
“seating  arrangements”,  IP  provided  as  document  for  discussion  before  the  b'me 

Emotional  tone 

of  the  event 

Discussion  a:  campuses  openly  discussed  their  frustrations/concerns  about 
services  / support  delivered  from  the  institutional  office.  Campus  participants  were 
clear  and  coristructive.  Some  institutional  officers  struggled  to  not  "defend”  them- 
selves.  VC  was  clear  only  clarifying  questions  asked.  It  was  a very  productive  session. 

Discussion  b:  discussion  of  IP  by  institutional  office.  Boring  session,  not  structured 
well,  not  very  productive. 

Participants 

Chairwas  the  VC  (1) 

Institutional  Management  (IM)  members  = VP,  campus  rectors,  directors  (7) 
Institutional  colleagues  reporting  to  IM  members  (about  20) 

Campus  management  representatives  (about  15)lnterpreters  (2) 

Message  form 

Mainly  spoken  SA  English  & Afrikaans.  Interpretation  available  from  Afrikaans  to 
English.  IP  written  in  English. 

Message  content 

(a)  Campuses  raise  their  frustrations  / concerns  about  services/support  delivered  by 
institutional  office. 

(b)  Institutional  office  discussed  campus  issues  & possible  action  plans  (after  campus 
representatives  left  the  meeting)  and  discussed  the  IP 

Nature  of  participation 

(a)  Campus  representatives  participated  openly,  fully,  effectively.  It  was  clear  that 
there  was  great  appreciation  for  the  opportunity  to  discuss  frustrations/concerns. 

A mix  of  Afrikaans/English  was  used. 

(b)  Institutional  office  discussion  of  IP  - “difficulf  to  participate: 

- because  one  had  to  ask  clarifying  questions  and  not  “defend"  and 

- because  all  members  do  not  know  IP  equally  well  & aim  was  unclear 

Perception  of  the 
effectiveness  of  the 

communicative  event 

During  the  event,  colleagues  mostly  used  English.  There  were  one  or  two  colleagues 
who  used  Afrikaans  throughout  the  session  as  well.  At  the  end  of  the  session,  when 
colleagues  reflected  on  the  success  of  the  meeting,  many  colleagues  started  to  use 
Afrikaans  all  of  a sudden.  Interpretation  services  were  available. 
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What  is  learned  about 

functional 

multilingualism? 

It  seems  that  many  Afrikaans  colleagues  are  becoming  used  to  discuss  relevant 
business  aspects  in  English.  However,  when  they  are  asked  to  reflect  on  the 
meeting,  many  of  the  colleagues  resorted  to  Afrikaans.  It  could  be  that  the  skill  of 
reflection  requires  a higher  order  thinking/cognitive  skill  and  that  Afrikaans 
colleagues  are  still  more  comfortable  in  Afrikaans  when  they  have  to  conduct  such 
a task? 

Other  comments 

A lot  of  English  used,  selected  and  announced  Afrikaans.End  of  meeting  when  VC 
requested  members  to  reflect  on  session,  a lot  of  Afrikaans. 

APPENDIX  D - Specific  communicative  event  St  participant  information: 


Type  of  event: 

Date: 

Venue: 

Nr  of  people: 

Name  1 (post): 

Name  2 (post) 
Relationship 
between  Name  1 & 

2 & Observer: 

Specific  communicative  event 

23-01-2006 

Transnet  (Potch) 

About  45 

Partídpant  1 - Gender:  Male;  Race:  White;  Age:  50+ 

Partídpant  2 - Gender:  Female;  Race:  Black;  Age:  50 

2 Reports  to  1 - Observer  reports  to  2 

Participant 

Information  about  dialogue: 

Field  notes  about  dialogue 

1 

Invited  all  participants  at  the  end  of  the  meeting  to  reflect  on  the  usefulness  of  the 
session.  Almost  all  participants  that  were  Afrikaans  used  Afrikaans,  contrary  to  their 
behaviour  earlier  in  the  meeting  where  many  of  them  used  English.  The  participant 
was  about  the  3,d  last  person  to  reflect. 

2 

1 have  great  appreciation  for  the  fact  that  we  all  have  different  mother  tongues.  1 also 
understand  that  people  use  their  mother  tongues  when  they  start  to  relax.  But  1 do 
want  to  seriously  request  members  to  please  keep  in  mind  that  we  do  not  all  share 
the  same  mother  tongue.  If  members  could  please  remember  to  speak  slowly  and 
clearly  and  choose  their  vocabulary  carefully  when  they  use  their  mother  tongues  in 
the  presence  of  non-mother  tongue  users,  it  will  be  appreciated. 

1 

Acknowledged  the  remark  and  agreed. 

Other  comments  / field  notes  / questions  to  ask  participant  in  interview 

a)  Ask  participant  to  relate  incident  first.  Then  check  if  my  field  notes  are  correct. 

b)  Explain  your  feelings  when  you  made  the  utterance. 

c)  Why  did  you  not  use  the  interpretation  services?  If  I remember  correctly,  you  did  not  even  take  a set 
of  ear  phones  at  the  beginning? 
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APPENDIX  E - Formal  planning  meeting  of  Extended  Senior  Management  of  NWU 


Type  of  event: 

Date: 

Venue: 

Nr  of  people: 
Role/relationship 
of  observer  in  event: 

Formal  planning  meeting  of  senior  management 

21-11-2005 

Crystal  Springs  conference  cenhe  (Rustenburg) 

About  45 

Member  of  extended  senior  management 

Category 

Field  notes 

Scene: 

Type  of  event:  Formal  strategic  planning  meeting  called  by  Vice-Chancellor 

Topic:  Strategic  priorities  for  2006 

Purpose:  An  opportunity  for  Extended  Senior  Management  of  the  NWU  to 

discuss  strategic  priorities  for  2006 

Setting:  Conference  venue  away  from  campus  where  all  colleagues  stayed 

for  2 days  (sleeping  over  for  1 night) 

Emotional  tone 

of  the  event 

It  was  the  meeting  of  this  nature  of  the  extended  senior  management  meeting, 

There  was  a sense  oí  excitement,  but  colleagues  were  also  still  getting  to  know  each 
other.  The  deans  from  the  Potchefstroom  campus  came  to  the  meeting  prepared  to 
take  stands  on  some  issues  which  they  regarded  as  important.  The  deans  from  the 

Mafikeng  campus  had  a very  successful  strategic  planning  session  for  research 
development  in  the  previous  week  and  also  communicated  a sense  of  shared  views 
about  certain  matters. 

Participants 

Chairwas  the  VC  (1) 

Institutional  Management  (IM)  members  = VP,  campus  rectors,  directors  (7) 

Institutional  colleagues  reporting  to  IM  members  (about  20) 

Campus  management  representatives  (about  1 5) 

Interpreters  (2) 

Message  form 

Mainly  spoken  SA  English  & Afrikaans. 

Interpretation  available  from  Afrikaans  to  English. 

Documentation  was  in  English, 

Message  content 

Due  to  the  confidentiality  of  these  types  of  meetings,  no  details  about  the  message 
content  will  be  shared.  In  general,  the  message  content  included: 

(a)  discussion  of  the  strategic  priorities  of  the  NWU  for  2006 

(b)  feedback  from  the  most  recent  Council  meeting 

(c)  feedback  from  the  “What  Works”  survey 

(d)  small  group  discussions  about  specific  elements  of  the  “What  Works"  survey 
to  discuss  interventions  / recommendations 

Nature  of  participation? 

In  general  there  was  a spirit  of  collaboration  and  co-operation, 

At  various  points,  the  deans  made  sure  that  they  drove  home  the  issues  they  believe 
are  important. 

Perception  of  the 
effectiveness  of  the 

communicative  event? 

During  the  event,  the  chairperson  continued  to  use  Afrikaans  and  English 
alternatively  for  consecutive  agenda  items  and  the  interpretation  services  were 
available  throughout  the  planning  session.  During  the  two  days,  it  became  clear 
that  some  Afrikaans  colleagues  struggle  to  express  themselves  eloquently  in  English. 

Choice  of  language  use  during  the  break  away  sessions  seemed  to  have  been 
problematic?  The  comments  made  by  the  participant  at  the  end  of  the  session  is 
proof  of  this,  as  well  as  the  observer's  own  discomfort  in  her  break  away  session. 

(In  the  observer’s  break  away  session  there  were  six  participants  - FV,  ALC,  MNT, 
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ASCVR,  TC,  MV  - and  only  one  of  the  participants  did  not  use  Afrikaans  as  a mother 
tongue.  During  the  session,  the  Afrikaans  colleagues  continually  spoke  Afrikaans, 
although  they  were  quite  aware  that  one  member  do  not  speak  Afrikaans  that  well 
and  that  there  were  no  interpretation  services.  The  observer  tried  to  interpret  some  of 
these  Afrikaans  statements  in  generai  so  that  one  member  would  not  be  excluded, 
but  this  was  an  “awkward"  thing  to  do,  so  she  did  not  interpret  all  that  was  said.) 

What  is  learned 

about  functional 
multilingualism? 

It  seems  that  small  group  communication  might  be  problematic  in  a context  where  an 
institution  adopted  a functional  multilingual  approach.  If  the  interpretation  service  is 
not  available  in  every  break  away  session,  it  seems  that  more  guidelinesflanguage 
table  manners"  need  to  be  developed  and  shared  so  that  all  participants  could 
benefit  from  the  small  group  discussion  as  well?  Communication  in  the  larger  groups, 
where  interpretation  services  were  available,  proceeded  with  apparently  more 
success? 

Other  comments? 

A lot  of  English  used,  selected  and  announced  Afrikaans.  The  communicative  event 
of  interest  occurred  at  the  end  of  the  meeting  when  participants  reflected  on  the 
success/effectiveness  of  the  meeting. 

APPENDIX  F - 

- Specific  communicative  event  sc  participant  information 

Type  of  event: 
Date: 

Venue: 

Nr  of  People: 
Participant  1: 
Participant  2 

Specific  communicative  event 

21  November  2005 

Crystal  Springs  Conference  Venue 

About  45 

Dean  of  a Faculty  - Gender:  Male;  Race:  ?;  Age:  ? 

Not  applicable  - Gender:  N/A;  Race:  N/A;  Age:  N/A 

Relationship 

Observer  supports  academic  managers  with  academic  staff 

between  participants  development  and  student  academic  development  matters  on 
and  observer  an  institutional  level 


Participant 

Information  about  dialogue: 

Field  notes  about  dialogue 

1 

1 do  not  want  to  upset  people  at  the  end  of  the  session.  But  1 feel  1 have  to  say 
something  about  language.  1 do  not  understand  why  people  who  can  speak  English 
insist  on  using  Afrikaans  in  this  meeting?  1 appreciate  the  interpretation  services  but 

1 struggle  to  understand  why  people  who  can  speak  English  insist  on  using  Afrikaans 
at  meetings?  1 was  a member  of  a smaller  group  yesterday  at  the  break  away 
sessions.  When  1 walked  into  the  room,  somebody  was  telling  a joke  in  Afrikaans. 

The  person  saw  me  enter  and  continued  to  tell  the  joke  in  Afrikaans.  1 experienced 
this  as  rude.  They  did  not  even  attempt  to  include  me  by  translating  the  joke. 

Other  comments  / field  notes  / questions  to  ask  participant  in  interview 

Note: 

The  chairperson  of  the  meeting  did  not  attempt  to  answer  the  participant/to  provide 
reasons/conclusions  to  the  discussion.  The  issue  was  raised  (as  many  other  issues 
were  at  the  end  of  the  session)  and  not  really  dealt  with  by  the  meeting. 

Questions  to 
participant: 

1 ) Did  1 capture  the  incident  correctly?  How  could  1 improve  the  accuracy? 

2)  What  did  you  feel  when  you  raised  the  issue? 

3)  Why  does  it  bother  you  that  colleagues  use  Afrikaans?  The  interpretation 
services  are  supposed  to  ensure  that  we  understand  what  is  said  at  the  meeting. 
What  is  your  opinion  about  the  use  of  the  interpretation  services? 

Test  efflciency  and  utility:  longer  or  shorter  tests 

]urie  Geldenhuys 
University  of  Pretoria 


Test  efficiency  and  utility:  longer  or  shorter  tests 

Questions  on  test  efficiency  and  utility,  though  often  neglected,  remain  critically 
important  considerations  for  test  design.  Conventional  wisdom  is  that  the  longer 
the  test,  the  more  reliable  it  potentially  is.  However,  while  tests  can  be  both  valid 
and  reliable  and  in  that  sense efficient,  if  they  are  too  long  that  might  make  them 
less  useful.  A test  has  low  utility  if  it  is  too  long  or  requires  elaborate  logistical 
and  administrative  arrangements. 

This  paper  will  explain  to  what  extent  using  a shorter  test  would  possibly 
limit  its  efficiency  on  the  one  hand,  while,  on  the  other,  make  it  more  useful.  This 
issue  will  befurther  elaborated  through  the  use  ofexamples  takenfrom  the  tests  of 
academic  literacy  levels  (TALL)from  the  University  ofPretoria.  The  method  used 
for  comparing  the  shorter  and  longer  versions  ofthese  tests  will  be  statistical 
measures  relating  to  the  reliability  (alpha)  ofeach  version.  An  argument  will  be 
proposed  ofhow  large  a drop  in  reliability  may  still  be  acceptable  for  a shorter  test. 


1 . Introduction 

If  we  characterize  language  tests  and,  more  specifically,  tests  of  academic  literacy 
as  applied  linguistic  instruments,  we  may  argue  that  they  need  to  conform  to  the 
conditions  that  apply  to  the  development  of  responsible  applied  linguistic  de- 
signs.  This  entails  that  language  tests  should  be  both  valid  and  reliable.  These 
conditions  are  indeed  necessary  for  such  tests,  as  is  their  theoreticai  defensibility 
or  so  called  construct  validity. 

Though  often  neglected,  questions  on  test  efficiency  and  utility  also  remain 
critically  important  considerations  for  test  design.  Conventional  wisdom  is  that 
the  longer  a test  is,  the  more  reliable  it  potentially  is.  However,  whiie  tests  can  be 
both  valid  and  reliable  and  in  that  sense  efficient,  if  they  are  too  long,  that  may 
make  them  less  useful.  A test  has  low  utility  if  it  is  too  long  or  requires  elaborate 
logistical  and  administrative  arrangements. 

The  aim  of  this  article  is  to  explain  whether  using  a shorter  test  would  possi- 
bly  limit  its  efficiency  on  the  one  hand,  while,  on  the  other  hand,  make  it  more 
useful.  This  issue  will  be  elaborated  with  reference  to  the  design  of  tests  to  meas- 
ure  academic  literacy  levels,  and  spedfically  the  focus  on  test  utility  that  present- 
ed  itself  when  we  started  developing  a postgraduate  test  of  academic  literacy 
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(TALPS)  for  use  at  the  University  of  Pretoria,  as  well  as  at  other  interested  academ- 
ic  institutions  for  higher  learning  in  South  Africa. 

The  aim  for  developing  TALPS  was  more  spesifically  to  test  the  academic 
literacy  of  postgraduate  students  who  enrol  at  the  University  of  Pretoria  for  a 
postgraduate  degree.  The  original  format  was  more  or  less  equivalent  to  that  of 
TALL  (test  for  academic  literacy  levels)  which  is  administered  to  first  year  stu- 
dents  at  the  beginning  of  each  academic  year  at  three  universities  in  South  Afri- 
ca.,  but  was  extended  to  include  a section  on  text  editing  as  well  as  a writing  task, 
both  of  which  are  not  included  in  TALL.  The  level  of  the  postgraduate  test  is  also 
higher  in  the  sense  that  it  includes  reading  passages  that  are  more  difficult  than 
those  in  TALL.  The  first  draft  of  the  test  comprised  of  173  items.  By  means  of  two 
rounds  of  piloting,  the  number  of  questions  was  first  reduced  to  150  items  and 
then  to  120  items.  The  intention  is  to  shorten  the  test  to  a final  draft  which  will 
eventually  count  out  of  100  marks  consisting  of  20  marks  for  a written  text  and  80 
marks  multiple  choice  questions. 


2.  Test  length 

The  length  of  a test  depends  on  its  purpose  and  the  statistical  properties  of  the 
items  (Owen  & Taljaard,  1996:22).  If  the  items  are  heterogeneous  more  items  are 
needed  than  when  the  items  are  of  a homogeneous  nature.  The  more  heterogene- 
ous  items  there  are,  the  less  reliable  the  test  potentially  becomes.  One  can  state 
here  that  in  the  case  of  tests  of  academic  literacy,  items,  all  being  language  related, 
probably  are  of  a more  homogeneous  nature,  in  a way  making  the  tests  more 
reliable.  As  Van  der  Slik  and  Weideman  (2005)  have  pointed  out,  however,  the 
richer  the  construct  of  a test,  the  more  heterogeneity  the  test  designer  may  have  to 
tolerate.  Compare,  for  example,  the  following  factor  analysis  of  one  of  the  pilots 
for  TALPS  (Figure  1). 

Here  it  is  evident  that  there  is  some  measure  of  heterogeneity:  items  73,  as  well 
as  62  - 66  seem  to  fall  outside  of  the  area  covered  by  the  others.  However,  overall 
the  reliability  of  the  test  (as  measured  in  terms  of  Cronbach's  alpha),  was  calculat- 
ed  at  0.85,  which  is  rather  high  for  a pilot.  In  light  of  the  factor  analysis  the  test 
items  are  not  as  heterogenous  as  one  may  have  expected.  Most  cluster  around  the 
right  end  of  factor  one.  This  means  that  the  items  are  more  homogenous  and 
therefore  probably  support  the  idea  of  a shorter  test.  Moreover,  the  length  of  a test 
also  depends  on  the  time  available  for  administering  the  test.  In  addition,  the 
form  and  content  of  the  items,  as  well  as  the  difficulty  level  and  the  time  needed 
to  read  the  items,  all  contribute  in  determining  the  length  of  a test  (Owen  & 
Taljaard,  1996:23). 
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TiaPlus  Factor  Analysis:  Subgroup  0 - Subtest  0 


Factor  1 

Figure  1:  Measure  of  homogeneity/heterogeneity  of  TALPS  first  pilot 


3.  Administering  tests 

Weideman  (2006a:  83)  points  out  that,  since  every  test  has  to  be  implemented,  its 
design  and  development 

anticipates  its  contextualization  within  some  social  environment,  and  the  way  it 
will  operate  and  regulate  the  interaction  between  test  designers,  test  adminis- 
trators,  test  takers  (testees),  administrative  officials,  lecturers,  and  others  in- 
volved.  This  is  the  social  dimension  that  is  unique  to  each  implementation  (or 
administration,  to  use  testing  terminology)  of  the  test,  and  it  expresses  for  this 
particular  case  the  relation  between  the  technical  and  social  aspects  of  our  world. 

At  the  beginning  of  each  academic  year  one  working  day  (eight  hours)  is  set  aside  to 
administer  the  test  of  academic  literacy  levels  (TALL)  to  approximately  4400  stu- 
dents  at  the  University  of  Pretoria  alone.  This  is  over  and  above  the  Afrikaans 
version,  TAG,  or  Toets  van  Akademiese  Geletterdheidsvlakke,  which  is  administered  to 
approximately  2600  students,  bringing  the  total  to  about  7000  students.  Administer- 
ing  a test  to  such  a number  of  students  is  a mammoth  task  and  places  a huge  admin- 
istrativeburden  on  the  personnel  involved.  Other  than  administrative  constraints, 
logistical  factors  play  a roll  as  well,  for  there  are  only  a limited  number  of  large 
venues  at  the  university  in  which  the  test  can  be  administered.  Weideman  (2006a: 
83;  2007b)  notes  that  there  are,  amongst  other  things,  a number  of  trade-offs  between 
the  technical  utility  of  a test  and  its  reliability.  A test  may  be  so  short  that  it  has  high 
utility,  but  perhaps  too  short  to  achieve  the  desired  level  of  reliability.  Thus: 
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The  utility  of  a test  requires  that  the  test  designer  should  carefully  weigh  a 
variety  of  potentially  conflicting  demands,  and  opt  not  only  for  the  socially 
most  appropriate,  but  also  for  a frugal  solution.  The  various  trade-offs  ...  that 
present  themselves  to  test  designers,  not  only  between  conflicting  sets  of  po- 
litical  interests,  but  also  between  reliability  and  utility  [generate  a need]  to 
weigh  or  assess,  to  harmonise  and  then  justify  a tough  and  responsible  techni- 
cal  design  decision.  (Weideman  2006a:  83-84) 

Taking  the  above  mentioned  into  consideration,  the  question  could  be  asked: 
could  a shorter  test  add  to  the  utility  of  the  test,  even  though  it  may  probably 
lessen  the  efficiency? 

Before  elaborating  further  on  this  issue,  I would  like  to  give  the  original  argu- 
ment  for  a shorter  test  in  the  case  of  TALPS.  When  we  started  constructing  a test 
for  postgraduate  students,  the  test  consisted  of  about  173  items  and  would  take 
students  approximately  two  and  a half  hours  to  complete.  Given  the  potentially 
heterogeneous  test  population  - students  that  register  for  postgraduate  studies  at 
the  University  of  Pretoria  vary  from  those  who  are  very  literate  in  English  to 
students  who  come  from,  for  example,  French-speaking  African  countries  and 
countries  in  Asia  and  who  are  not  that  literate  in  English  - we  needed  an  instru- 
ment  to  give  a first,  rough  indication  of  level  in  order  not  to  waste  the  time  of 
those  competent  enough  to  make  a success  of  their  studies.  The  line  of  thought 
therefore  was  to  administer  a shorter  test  to  all  candidates,  and  after  receiving  the 
results  of  this  test  administer  the  longer  test  to  candidates  who  were  potentially  at 
risk  of  not  completing  their  studies  in  the  desired  time. 

To  answer  the  question  of  whether  a shorter  test  is  possible,  I will  in  the  first 
place  give  a summary  of  the  grounds  on  which  our  test  of  academic  literacy  levels 
is  conventionally  set,  and  the  components  that  usually  make  up  the  test.  Then  I 
will  state  an  argument  for  omitting  some  of  these  components  in  order  to  shorten 
the  test.  Having  taken  a decision  on  which  components  to  consider  using  for  a 
shorter  test,  these  components  will  be  correlated  with  one  another  and  with  the 
test  total.  The  initial  thinking  was  that,  if  there  is  a good  correlation  between 
some  of  the  components  and  the  test  total,  it  could  be  argued  that  the  test  could  be 
shortened  by  including  only  these  specific  subtests. 


4.  Validity,  reliability  and  the  construct 

According  to  Weideman  (2006a:  74),  language  testers  traditionally  have  attempt- 
ed  to  ensure  faimess  in  language  testing  by  designing  tests  that  are  both  valid  and 
reliable.  Henning  (1987:89)  defines  validity  as  follows: 
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Validity  in  general  refers  to  the  appropriateness  of  a given  test  or  any  of  its 
component  parts  as  a measure  of  what  it  is  purported  to  measure.  A test  is  said 
to  be  valid  to  the  extent  that  it  measures  what  it  is  supposed  to  measure.  It 
follows  that  the  term  valid  when  used  to  describe  a test  should  usually  be 
accompanied  by  the  preposition  for.  Any  test  then  may  be  valid  for  some 
purposes,  but  not  for  others. 

There  are  different  types  of  validity,  such  as  empirical  validity,  face  validity  and 
construct  validity. 

Empirical  validity  is  ensured  by  setting  strict  statistical  parameters  for  the 
various  items  in  a test.  Items  should  only  become  part  of  a test  after  a reasonable 
amount  of  piloting  which  ensures  that  each  item  measures  what  it  is  supposed  to 
measure.  It  must  therefore  discriminate  well  between  those  whose  total  scores  fall 
into  the  top  quartile  and  those  whose  total  scores  fall  into  the  bottom  group.  The 
parameter  for  this  discrimination  value  of  an  item  is  normally  set  between  0.3  and 
1 on  an  index  from  0 to  1.  To  arrive  at  the  specific  values  for  each  test  item  test 
designers  should  trial  and  evaluate  each  item  as  well  as  the  test  as  a whole. 

The  face  validity  of  a test  refers  to  the  way  in  which  it  impresses  or  fails  to 
impress  a lay  person,  but  also  the  way  in  which  the  test  appears  to  be  valid  for  the 
users  of  the  test.  Tests  that  do  not  appear  to  be  valid  to  users  may  not  be  taken 
seriously  for  their  given  purpose.  If,  however,  test  takers  consider  a test  to  have 
face  validity,  they  are  more  likely  to  perform  to  the  best  of  their  ability  and  re- 
spond  appropriately  to  items  (Alderson,  Clapham  & Wall,  1995;  Weideman,  2006a). 

A test  should  also  have  construct  validity.  The  latter  is  an  analysis  that  indi- 
cates  whether  the  theory  or  analytical  definition  or  construct  that  the  test  design 
is  built  upon,  is  valid.  Alderson,  Clapham  and  Wall  (1995:183)  state  that  construct 
validity  can  be  explained  as  follows: 

The  term  construct  refers  to  a psychological  construct,  a theoretical  conceptu- 
alization  about  an  aspect  of  human  behaviour  that  cannot  be  measured  or 
observed  directly...  Construct  validation  is  the  process  of  gathering  evidence 
to  support  the  contention  that  a given  test  indeed  measures  the  psychological 
construct  the  makers  intend  it  to  measure. 

The  Test  of  Academic  Literacy  for  Postgraduate  Students  (TALPS)  that  is  being 
developed  at  the  University  of  Pretoria  sets  out  to  measure  academic  literacy  in 
terms  of  a definition  (Weideman,  2007a:  xi)  that  assumes  that  students  who  are 
academically  literate  should  be  able  to: 

• understand  a range  of  academic  vocabulary  in  context; 

• interpret  and  use  metaphor  and  idiom,  and  perceive  connotation,  word 
play  and  ambiguity; 
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• understand  relations  between  different  parts  of  a text,  be  aware  of  the 
logical  development  of  (an  academic)  text,  via  introductions  to  conclu- 
sions,  and  know  how  to  use  language  that  serves  to  make  the  different 
parts  of  a text  hang  together; 

• interpret  different  kinds  of  text  type  (genre),  and  show  sensitivity  for  the 
meaning  that  they  convey,  and  the  audience  that  they  are  aimed  at; 

• interpret,  use  and  produce  information  presented  in  graphic  or  visual 
format; 

• make  distinctions  between  essential  and  non-essential  information,  fact 
and  opinion,  propositions  and  arguments;  distinguish  between  cause  and 
effect,  classify,  categorise  and  handle  data  that  make  comparisons; 

• see  sequence  and  order,  do  simple  numerical  estimations  and  computa- 
tions  that  are  relevant  to  academic  information,  that  allow  comparisons  to 
be  made,  and  can  be  applied  for  the  purposes  of  an  argument; 

• know  what  counts  as  evidence  for  an  argument,  extrapolate  from  informa- 
tion  by  making  inferences,  and  apply  the  information  or  its  implications 
to  other  cases  than  the  one  at  hand; 

• understand  the  communicative  function  of  various  ways  of  expression  in 
academic  language  (such  as  defining,  providing  examples,  arguing);  and 

• make  meaning  (e.g.  of  an  academic  text)  beyond  the  level  of  the  sentence. 

The  critical  component  of  the  above  turns  out  to  be  the  ability  to  use  language  in 
academic  discourse  that  enables  one  to  compare  and  classify,  categorise  and  make 
distinctions  between  essential  and  non-essential  information. 

From  an  applied  linguistics  point  of  view,  it  should  be  noted  that  test  design- 
ers  use  the  definition  of  academic  literacy  given  above  as  a justification  for  the 
various  task  types  that  are  included  in  the  test,  and  as  a rationale  for  why  the  test 
is  made  up  in  a certain  way.  Therefore  the  first  draft  of  TALPS  is  made  up  of  the 
following  task  types:  scrambled  text;  interpreting  and  understanding  visual  and 
graphic  information;  dictionary  definitions,  register  and  text  type;  understand- 
ing  texts  (longer  reading  passages);  academic  vocabulary;  text  editing  and  gram- 
mar  and  text  relations.  The  justification  for  using  the  above  mentioned  text  types 
is  normally  further  articulated  in  the  form  of  a set  of  specifications  for  each  text 
type.  Van  Dyk  and  Weideman  (2004b)  present  a table  in  which  each  task  type  is 
matched  with  a component  or  components  of  the  construct: 
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Table  1:  Specificationsandtasktypes:  TALL 


Specification 

(component  of  construct) 

Task  type(s)  measuring  / potentially  measuring 
this  component 

Vocabulary  comprehension 

Vocabulary  knowledge  test,  longer  reading  passages, 

text  editing 

Understanding  metaphor  & idiom 

Longer  reading  passages 

Textuality  (cohesion  and  grammar) 

Scrambled  text,  text  editing,  (perhaps)  register  and  text  type, 
longer  reading  passages,  academic  writing  tasks 

Understanding  text  type  (genre) 

Register  and  text  type,  interpreting  and  understanding  visual 
and  graphic  information,  scrambled  text,  text  editing,  longer 
reading  passages,  academic  writing  tasks 

Understanding  visual  & graphic 

information 

Interpreting  and  understanding  visual  and  graphic  information, 
(potentially)  longer  reading  passages 

Distinguishing  essential/non-essential 

Longer  reading  passages,  interpreting  and  understanding  visual 
and  graphic  information,  academic  writing  tasks 

Numerical  computation 

Interpreting  and  understanding  visual  and  graphic  information, 
longer  reading  passages 

Extrapolation  and  application 

Longer  reading  passages,  academic  writing  tasks,  (potentially) 
interpreting  and  understanding  visual  and  graphic  information 

Communicative  function 

Longer  reading  passages,  (possibly  also)  text  editing, 

scrambled  text 

Making  meaning  beyond  the  sentence 

Longer  reading  passages,  register  and  text  type,  scrambled  text, 

interpreting  and  understanding  visual  and  graphic  information 

The  following  table  illustrates  the  selected  task  types  as  well  as  the  marks  allocat- 
ed  to  each  task  type  for  the  first  two  drafts  of  TALPS.  In  comparison  with  the  task 
types  selected  for  TALL,  two  additional  task  types  were  selected,  i.e.  dictionary 
definitions  and  grammar  and  text  relations. 
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Table  2:  Task  types  and  marks  allocated  to  them  in  two  TALPS  draft  versions 


Task  type 

Marks  (First  draft) 

Marks  (Second  draft;  pilot) 

Scrambled  text 

15 

5 

Graphic  and  visual  literacy 

16 

16 

Dictionary  definitions 

5 

5 

Academic  vocabulary 

40 

27 

Text  type 

5 

5 

Understanding  texts 

60 

60 

Grammar  and  text  relations 

22 

22 

Text  edit 

10 

10 

Total 

173 

150 

As  can  be  seen  from  the  table  above  there  is  but  a slight  difference  between  the 
first  and  the  second  draft  of  TALPS  as  far  as  the  marks  allocated  to  the  different 
task  types  are  concerned.  This  happened  as  a result  of  the  number  of  items  that 
were  reduced  for  two  of  the  task  types,  and  was  the  first  attempt  to  shorten  TALPS. 

In  principle,  a test  cannot  be  valid  unless  it  is  reliable.  (Alderson  et  al.  1995:18 7). 
For  a test  designer  to  develop  a fair  test,  different  kinds  of  validity  are  not  the  only 
important  factors.  The  test  also  requires  reliability.  Test  makers  therefore  focus  on 
the  internal  reliability  or  consistency  of  a test.  We  have  noted  one  such  measure  in 
the  factor  analysis  (Figure  1 above),  which  shows  the  extent  to  which  a test  is 
heterogeneous  or  homogeneous  in  what  it  measures.  The  reliability  of  a test  is 
usually  expressed  in  terms  of  such  statistical  measures.  In  the  case  of  the  intemal 
reliability  of  a test,  i.e.  its  consistency  across  all  items  in  the  test,  this  statistical 
measure  is  generally  expressed  in  terms  of  an  index  (from  0 to  1)  called  alpha. 
Such  a reliability  index  gives  an  indication  of  how  internally  consistent  a test  is. 
The  alpha  for  high  stakes  tests  is  considered  to  be  at  least  0.6,  but  preferably 
higher  than  0.7.  Although  we  do  not  yet  know  whether  TALPS  will  be  employed 
as  a high  or  medium  stakes  test,  we  are  pleased  with  the  reliability  of  the  pilots  of 
the  test  as  a whole,  and  certain  components  of  it,  which  we  have  so  far  tested: 


Ensovoort:  ]aargang  1 1 , nommer  2,  2007 


79 


Table  3:  TALPS:  Reliability  measures 


Date  and  component  of  the  test 

Alpha 

May  2007  (longer  draft) 

0.85 

May  2007  (Graphic  & visual  literacy) 

0.69 

May  2007  (Text  type  & text  edit) 

0.69 

May  2007  (Scrambled  text) 

0.78 

Average 

0.75 

In  addition  to  being  as  reliable  as  one  would  wish  for,  for  such  shorter  tests  (of 
between  10  and  15  marks,  for  the  latter  three  mentioned  above),  there  is  also 
evidence  that  the  spread  of  marks  achieved,  i.e.  the  way  that  the  test  manages  to 
discriminate  between  testees,  is  highly  satisfactory.  Compare  (Figure  2)  the  way 
in  which  the  TALPS  visual  and  graphic  literacy  component  test  pilot  discrimi- 
nated  among  the  551  testees  to  which  it  was  administered,  in  the  following  score 
distribution  table  generated  by  means  of  an  Iteman  analysis  of  the  results: 
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Figure  2:  Score  distribution  table  - TALPS  pilot  (graphic  & visual  literacy) 
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Given  these  indications  of  reliability  and  the  discriminative  power  of  the  draft 
TALPS  and  some  of  its  subcomponents,  we  can  again  ask  whether  a shorter  test  is 
a viable  option.  The  first  answer  to  this,  as  has  been  stated  before,  is  that  although 
a test  can  be  valid  and  reliable,  its  utility  can  still  be  undermined  if  it  is  too  long. 


5.  An  alternative  for  the  conventional  format 

When  one  has  the  intention  of  shortening  a test  of  academic  literacy  levels,  the 
first  question  that  comes  to  mind  is:  Which  components  will  one  keep  and  which 
are  going  to  be  omitted?  To  establish  the  latter,  subtest  intercorrelations  were 
done  on  the  results  of  the  2007  administration  at  the  University  of  Pretoria  of  our 
conventional  test  (TALL).  The  results  indicate  that,  while  the  correlation  between 
the  various  subtests  and  the  total  test  were  satisfactorily  high  (above  0.6  in  all  but 
two  of  the  cases,  and  in  the  latter,  <0.44),  the  inter-subtest  correlations  were  (since 
each  subtest  hypothetically  is  supposed  to  tap  into  a different  component  of  the 
construct)  as  desired,  viz.  below  0.5  in  all  but  three  of  the  cases.  The  complete  set 
of  inter-subtest  correlations  is  shown  in  Table  4. 

Table  4 shows  that  the  three  subtests  with  the  highest  correlation  are  under- 
standing  texts,  academic  vocabulary  and  text  editing.  The  question  now  arises. 


Table  4:  Subtest  intercorrelations 


Total 

Subtest 

Test 

Subtest(s) 

1 

2 

3 

4 

5 

6 

Scrambled  text 

1 

0.46 

Graphic  & visual  literacy  2 

0.67 

0.25 

Text  types 

3 

0.44 

0.14 

0.24 

Understanding 

4 

0.87 

0.33 

0.55 

0.34 

Academic  vocabulary 

5 

0.82 

0.29 

0.47 

0.33 

0.69 

Text  editing 

6 

0.84 

0.25 

0.44 

0.26 

0.59 

0.62 

Number  of  testees 

3905 

3905 

3905 

3905 

3905 

3905 

3905 

Number  of  items 

65 

5 

8 

5 

22 

9 

16 

Average  test  score 

39.87 

3.24 

5.54 

2.93 

14.72 

5.41 

8.03 

Standard  deviation 

13.35 

1.86 

2.19 

1.28 

4.57 

2,61 

5.15 

SEM 

3.34 

0.73 

1.11 

0.83 

1.93 

1.16 

1.44 

Average  P-value 

61.33 

64.79 

69.25 

58,59 

66.90 

60.11 

50.18 

Coefficient  Alpha 

0.94 

0.85 

0.74 

0.58 

0.82 

0,80 

0.92 

GLB 

0.97 

0.93 

0.81 

0.77 

0.85 

0.82 

0.96 

Asymptotic  GLB 

0.96 

0.93 

0.80 

0.77 

0.84 

0.82 

0.96 
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will  they  be  able  to  do  the  trick  on  their  own?  If  scores  of  only  these  three  are 
taken,  there  is  a high  correlation  with  the  original  total.  However,  this  high  cor- 
relation  is  to  be  expected,  since  the  three  mentioned  subtests  make  up  the  greater 
part  of  the  initial  test.  And  this  is  exactly  where  the  argument  of  using  these  three 
as  the  subtests  in  a shorter  test  runs  up  against  a wall. 

As  an  alternative,  to  see  whether  this  impediment  can  be  overcome,  we  ex- 
plored  doing  partial  correlations.  If  partial  correlations  are  done,  i.e.  analyses  in 
which  we  hold  steady  the  influence  of  some  parts  (in  this  case  subtests  1, 2,  and  3), 
then  there  is  indeed  some  relationship  (some  25%)  between  4, 5 and  6,  the  three 
subtests  that  could  potentially  constitute  a shorter  test  (see  Table  5). 


Table  5:  Subtest  partial  correlations  between  subtests  4,  5 and  6 


Subtest  4 

Subtest  5 

Subtest  6 

Subtest  4 

1.00000 

0.54 

0.42  ' 

(<.0001) 

(<.0001) 

Subtest  5 

0.54 

1.00000 

0.48 

(<.0001) 

(<.0001) 

Subtest  6 

0.42 

0.48 

1.00000 

(<.0001) 

(<.0001) 

Pearson  Partial  Correlation  Coefficients,  (probablility)  [N  = 3905] 


The  main  problem,  however,  is  that  even  these  analyses  are  unlikely  to  give  us 
empirical  answers  to  the  question  if  each  subtest  does  not  have  the  same  weight  as 
each  of  the  others. 


6.  Conclusion 

The  sensible  conclúsion  is  that  one  should  thus  either  change  the  design  of  the 
test,  or  acknowledge  that  one  has  to  make  the  decision  on  other  than  purely 
empirical  grounds.  This  means,  in  effect,  that  even  this  kind  of  decision  (about 
what  should  make  up  a shorter  test)  is,  in  the  end,  one  that  must  be  done  judi- 
ciously,  i.e.  after  weighing  up  various  issues.  Even  though  empirical  data  and 
analyses  may  provide  some  support  for  responsible  decisions,  this  may  not  al- 
ways  be  the  case. 

If  one  considers  the  use  to  which  TALPS  will  be  put,  namely  to  determine  the 
level  of  risk  that  a postgraduate  student  has  in  respect  of  academic  literacy,  a more 
productive  future  line  of  investigation  may  be  to  investigate  which  component(s) 
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(subtest(s))  has/have  the  better  predictive  validity,  i.e.  correlates  high  or  low  with 
some  other  measure  of  academic  success  (like  finishing  a postgraduate  degree  in 
time).  This  kind  of  validation,  against  another,  external  measure,  is  in  any  event 
desirable.  This  is  where  I hope  the  subsequent  research  should,  and  will,  take  us. 
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Moving  to  more  than  editing:  Standardised  feedback  in  practice 

This  article  reports  on  an  experiment  which  tested  how  effectively  standardised 
feedback  could  be  used  when  marking  L2  student  writing.  The  experiment  was 
conduded  by  using  a custom-programmed  software  tool  and  a set  of  standardised 
feedback  comments.  The  results  ofthe  experiment  prove  that  standardised feedback 
can  be  used  consistently  and  effectively  to  a degree,  even  though  some  refinements 
are  still  needed.  Using  standardised feedback  in  a standard  marking  environment 
can  assist  markers  in  raising  their  awareness  oferrors  and  in  more  accurately 
identifying  where  students  lack  knowledge.  With  some  refinements,  it  may  also  be 
possible  to  speed  up  the  marking  process. 


1 . Introduction  and  background  to  the  project 

The  process  of  providing  feedback  (marking)  on  student  essays  is  usually  very 
time-consuming.  Comparing  the  amount  of  time  spent  on  it  by  teachers  and  the 
amount  of  attention  paid  to  it  by  students,  it  may  be  considered  one  of  the  least 
effective  duties  of  language  teachers  (Moletsane,  2002:21;  Hyland,  1990:282). 

In  2004  a project  commenced  at  the  North-West  University  to  investigate  the 
possibility  of  gettingmore  "teaching"  out  of  marking.  The  main  objectives  of  the 
study  were  to: 

a.  establish  whether  standardised  feedback  would  ensure  more  clarity  for 
the  student  (Louw,  2006); 

b.  create  a system  to  keep  effective  records  of  student  development  (cf.  Wible, 
Kiu,  Chien,  Liu  & Tsao,  2001;  Louw,  2006); 

c.  establish  whether  standardised  feedback  could  be  used  consistently; 

d.  establish  whether  standardised  feedback  would  ensure  ease  of  use  for  the 
marker;  and 

e.  force  students  to  pay  attention  to  the  feedback. 

Spencer  (1998)  researched  strategies  in  responding  to  student  writing,  while  Wible 
et  al  (2001)  created  an  electronic  marking  system  used  to  keep  track  of  student 
development.  Wible  et  al'  s (2001)  system  did  not  work  with  standardised  feed- 
back,  while  Spencer  found  that  with  current  working  limitations,  certain  mark- 
ing  strategies  workbetter  than  others.  The  project  at  the  North-West  University 
aimed  to  integrate  these  findings  into  one  project.  The  first  step  was  to  establish 
whether  or  not  feedback  could  be  standardised  to  an  extent  and  if  it  would  actu- 
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ally  benefit  the  student.  In  Lou  w (2006)  this  was  found  to  be  the  case.  This  article 
reports  on  objectives  C and  D as  discussed  above.  In  addition,  the  feasibility  of  the 
marking  system  is  also  addressed. 

For  background  purposes,  a quick  overview  of  the  findings  with  regard  to 
feedback  is  presented  in  the  next  section. 


2.  What  is  effective  feedback? 

To  provide  standardised  feedback,  it  was  firstly  necessary  to  establish  exactly 
what  constitutes  feedback  as  well  as  the  nature  of  effective  feedback.  The  different 
classifications  of  feedback  are  too  numerous  and  intricate  to  discuss  here  (see 
Lou  w,  2006)  but  some  of  the  important  facts  about  feedback  can  briefly  be  summa- 
rised  as  follows: 

• The  interpretation  and  use  of  the  concept  "feedback"  is  closely  related  to 
the  user's  definition  of  and  attitude  to  "error". 

• Feedbackis  not  just  error  correction,but  any  response  (positive  or  nega- 
tive)  on  a student  textby  any  reader  of  the  text  (Hyland,  2003, 1990, 1998; 
Lyster  & Ranta,  1997;  Askew  & Lodge,  2000).  Per  definition  then,  feedback 
can  be  provided  in  many  different  ways. 

• Depending  on  the  purpose  or  background  of  the  'feedback  giver ',  feed- 
back  can  be  classified  as  performing  many  different  functions: 

- evidence  (linguists), 

- repair  (discourse  analysis), 

- correction  (L2  teachers),  and 

- focus  on  form  (SLA  researchers)  (Lyster  & Ranta,  1997:38). 

• Leamers  expect  feedback,  but  often  neglect  to  look  at  it  (Ferris,  2002:13-14). 

• There  are  many  advantages  and  disadvantages  to  feedback,  but  there  are 
conflicting  research  findings  regarding  these  (see  Ferris,  2003:127  and  Louw, 
2006). 

Because  there  are  so  many  overlapping  and  contrasting  definitions  of  feedback 
(see  Louw,  2006),  the  following  working  definitions  will  be  used  in  this  article: 

• An  error  is  any  instance  in  a text  which  is  incorrect  language  use  or  lan- 
guage  use  which  is  not  inherently  wrong,  but  which  could  be  improved. 

• Feedback  is  any  reaction  to  a text  by  any  reader  of  the  text,  for  the  purpose 
of  pointing  ou  t errors  to  the  writer.  In  keeping  with  the  definition  of  "er- 
ror  ",  feedback  could  also  indicate  satisfaction  with  something  in  the  text. 
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3.  Can  feedback  be  standardised? 

In  Louw  (2006)  a standardised  set  of  feedback  tags  was  created. 

The  tag  set  contains  a list  of  "popular  " error  tags  used  by  markers 
(as  established  by  research  and  in  an  experiment  in  Louw,  2006) 
and  in  corpus  linguistics  (Granger  & Meunier,  2003).  The  tag  set  is 
constantly  being  updated  and  refined.  An  example  of  the  current 
version  (at  the  time  of  writing)  is  attached  as  Addendum  A.  An 
experiment  proved  that  marking,  using  this  set  of  standardised 
feedback  comments,  is  more  effective  than  the  normal  squiggles, 
lines,  strike-throughs  and  question  marks  often  used  by  lecturers 
(Louw,  2006).  For  reference  purposes,  these  squiggles  and  other 
feedbackmarks  are  called  hieroglyphics.  See  Figure  1 for  an  exam- 
ple  of  hieroglyphics. 

In  the  experiment  in  Louw  (2006),  a hieroglyphic  marking  tech- 
nique  was  pitted  against  standardised  feedback  and  a blank  text 
on  which  the  number  of  errors  were  indicated,  but  no  errors  were 
marked.  The  results  indicated  that  students  were  seldom  able  to 
identify  errors  in  the  blank  texts,  much  less  correctly  revise  them. 

On  the  other  hand,  students  were  able  to  correctly  revise  errors 
rnarked  with  the  hieroglyphic  feedback,  but  the  standardised  feed- 
back  proved  to  deliver  the  greatest  improvement  in  all  categories 
tested.  This  shows  that  students  are  often  able  to  revise  errors  once  indicated  to 
them,  but  in  order  to  facilitate  maximal  irnprovement  in  writing,  standardised 
feedback  is  more  effective. 

Based  on  these  findings,  the  question  then  arises  whether  standardised  feed- 
back  could  be  implemented  in  practice  with  consistency.  Another  experiment 
was  therefore  conducted  to  investigate  the  following  questions: 

a Can  standardised  feedback  be  us.ed  consistently  by  markers? 

b.  Will  standardised  f eedback  make  it  easier  for  markers  to  mark  texts  effecti vely ? 

The  rest  of  this  paper  reports  on  this  experiment. 


Figure  1:  Example  of  hieroglyphic 
feedback 


4.  Methodology 

Four  markers  (two  experienced  and  two  inexperienced)  were  instructed  to  mark 
a number  of  L2  leamer  essays  using  the  feedback  tag  set  as  tool.  The  essays  for  the 
experiment  come  mostly  from  the  Tswana  Learner  English  Corpus  (TLE)  (Van 
Rooy  & Schaefer,  2002),  with  a minimal  number  of  essays  from  the  Afrikaans 
Learner  English  Corpus  which  is  still  under  construction. 

The  tag  set  was  incorporated  into  a custom-built  software  package.  The  soft- 
ware  package  imported  the  text  to  be  rnarked  into  a marking  window  and  had  the 
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entire  standardised  feedback  tag  list  in  a drop-down  tree  view  on  the  right  of  the 
text  (see  Figure  2).  When  finding  an  error,  markers  had  to  simply  highlight  the 
error  and  click  on  the  relevant  error  category.  The  computer  would  then  insert 
the  feedback.  Markers  were  shown  how  the  system  worked  but  were  not  given 
any  additional  instruction  on  how  to  mark  or  on  what  to  provide  feedback. 
Additional  feedback  comments  not  covered  by  the  tag  set  could  also  be  added  by 
making  use  of  the  comment  box.  It  should  be  noted  that  the  drop-down  tree  view 
simply  indicates  the  "name"  of  the  specific  error  tag  used  by  the  markers,  and  is 
not  the  (more  complete)  feedback  a student  would  see  (compare  Addendum  A 
and  Figure  3). 


Student  essay  Identified  error 


Drop-down  tree  view  of 
standardised  error  tag  set 


Relevant  error  tag  for 
the  highlighted  error 
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Figure  2:  Essay  marker  screenshot 
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Comment  box  in  case  markers 
need  to  provide  feedback  not  in 
the  standardised  tag  set 
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Student  errors  are 
highlighted  in  colour 


When  a student  moves 
his  / her  mouse  over  the 
highlighted  area,  a pop-up 
block  explains  the  error 


Figure  3:  Student  view  illustrating  how  students  will  receive  feedback 


Figure  2 illustrates  the  view  of  the  marking  systern  as  the  rnarker  would  see  it.  In 
this  figure,  the  marker  has  identified  a concord  error  and  highlighted  the  error. 
The  marker  has  also  identified  the  relevant  error  tag  (in  this  instance  "concord") 
in  the  tree  view  on  the  right  hand  side  of  the  screen. 

Figure  3 illustrates  how  the  student  would  receive  this  specific  feedback.  The 
student  will  get  his/her  text  back  in  an  HTML-file  which  can  be  opened  by  any 
standard  web  browser  like  tnternet  Explorer  or  Mozilla  Firefox.  Errors  are  high- 
lighted  in  different  colours,  for  example  red  indicates  a grammar  error.  In  this  case, 
the  student  has  moved  his/her  cursor  over  the  highlighted  word  and  got  the  pop- 
up  message,  "The  form  of  your  verb  should  agree  with  the  subject  it  refers  to". 

The  marked  essays  were  stored  electronically  with  their  error  tags  for  analysis. 
The  error  tags  the  markers  used  were  extracted  from  the  database.  All  the  markers 
did  not  mark  an  equal  number  of  essays.  To  counter  this  problem  and  in  order  to 
compare  apples  with  apples,  the  data  was  normalized.  The  number  of  times  a tag 
was  used  was  reworked  to  the  number  of  tímes  used  per  1000  errors  tagged.  All 
numbers  reported  in  this  paper  therefore  refers  to  the  normalized  totals. 

In  addition  to  analysing  the  tags  used,  the  markers  were  also  sent  a question- 
naire  with  18  questions  asking  them  about  their  experiences  with  the  marking 
system.  These  questions  are  included  in  Addendum  B. 
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The  research  aimed  to  answer  two  questions: 

1.  Can  standardised  feedback  be  used  consistently? 

2 Is  the  system  easy  to  use  for  the  markers  and  if  not,  how  can  it  be  im- 
proved? 

To  answer  question  one,  two  types  of  analysis  were  done  on  the  marked  essays. 

• An  analysis  of  all  the  tags  was  done  to  establish  marker  tendencies. 

• A close  analysis  of  the  way  the  markers  used  the  tag  "better  word"  was 
conducted.  (The  tag  was  chosen  because  all  four  of  the  markers  used  it  as 
one  of  their  top  ten  favourite  tags.) 

The  answers  to  the  questionnaire  were  also  used  to  judge  the  consistency  with 
which  the  markers  marked.  To  answer  question  two  regarding  the  ease  of  use  of 
the  marker  system,  the  answers  provided  on  the  questionnaire  were  used. 


5.  Results:  can  standardised  feedback  be  used  consistently? 

Marker  tendencies 

Table  1 indicates  the  top  20  tags  used  by  the  markers.  The  first  three  columns 
identify  and  explain  the  error  tag  (see  Addendum  A for  additional  clarification), 
while  'Knorm',  'Mnorm',  'Pnorm'  and  'Tnorm'  reveal  the  number  of  times  the 
specific  tag  was  used  by  the  different  markers.  The  column  "normed  total"  indi- 
cates  the  total  number  of  times  a tag  had  been  used  out  of  a total  of  4000  marked 
errors  (see  Table  1). 

With  regard  to  the  top  20  tags  shown  in  Table  1,  the  following  points  are 
interesting  to  note: 

1.  Four  of  the  top  20  are  errors  which  are  only  present  in  writing:  Punctua- 
tion  wrong,  Punctuation  missing,  Capitalization,  Spelling/typing  error. 
These  are  surface  elements  errors  only. 

2.  Seven  errors  have  to  do  with  lexis:  Wrong  word,  Better  word,  Word  form 
wrong,  Article  missing,  Preposition  wrong,  Word  choice  obscuring  mean- 
ing,  Determiner  incorrect. 

3.  Three  error  tags  have  to  do  with  morphology:  Concord,  Tense,  Omission 
plural  marker. 

4.  Three  errors  have  to  do  with  syntax:  Superfluous  general,  Omission  gen- 
eral,  Omission  verb. 

5.  Only  two  error  tags  deal  with  coherence  or  cohesion  and  both  of  these  are 
coherence  on  a small  scale  - within  paragraphs  or  within  sentences.  The 
relevant  tags  are:  Reasoning  inconclusive  and  Reference  vague. 
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Tabie  1 : Top  20  tags  used 


Subordinate 

Domain 

Set 

(error  tag) 

Knorm 

Mnorm 

Pnorm 

Tnorm 

Normed 

Total 

1 Presentation 

Spelling 

Spelling  / 
typing  error 

105 

92 

150 

62 

409 

2 Grammar 

Lexis 

Wrong  word 

92 

40 

87 

47 

266 

3 Presentation 

Capitalization 

Capitalization 

18 

130 

33 

25 

206 

4 Grammar 

Syntax 

Superfluous 

general 

53 

35 

26 

59 

173 

5 Grammar 

Morphology 

Concord 

19 

46 

79 

25 

169 

6 Presentation 

Punctuation 

Punctuation 

missing 

23 

44 

47 

51 

166 

7 Grammar 

Lexis 

Better  word 

8 

16 

81 

34 

139 

8 Grammar 

Syntax 

Omission 

general 

40 

26 

17 

32 

115 

9 Grammar 

Morphology 

Tense 

42 

29 

5 

38 

114 

10  Discourse 

Coherence 

Reasoning 

inconclusive 

73 

2 

8 

30 

113 

11  Grammar 

Lexis 

Word  form 
wrong 

21 

36 

40 

16 

112 

12  Grammar 

Morphology 

Omission 
plural  marker 

14 

61 

6 

29 

109 

13  Grammar 

Lexis 

Article  missing 

2 

62 

7 

19 

90 

14  Grammar 

Lexis 

Preposition 

wrong 

15 

16 

35 

23 

89 

15  Discourse 

Style 

Sentence 

vague 

53 

2 

23 

10 

88 

16  Grammar 

Lexis 

Determiner 

incorrect 

18 

39 

2 

26 

85 

17  Discourse 

Factual 

correctness 

Facts  wrong 

14 

2 

8 

44 

69 

18  Presentation 

Punctuation 

Punctuation 

wrong 

25 

5 

28 

10 

68 

19  Discourse 

Cohesion 

Reference 

vague 

9 

18 

0 

33 

61 

20  Grammar 

Lexis 

Word  choice 

obscuring 

meaning 

15 

28 

9 

7 

59 

20  Grammar 

Syntax 

Omission  verb 

4 

31 

10 

14 

59 

90 
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The  results  indicate  that  the  markers  did  not  only  focus  on  surface  elements. 
Style,  coherence  and  the  accuracy  of  facts  feature,  but  only  on  a small  scale  - 
within  the  paragraph  or  sentence  level.  These  results  are  the  same  for  all  four 
markers.  Although  it  is  not  ideal  to  focus  on  surface  structure  errors,  these  results 
are  no  different  from  those  found  in  previous  studies  (see  Louw,  2006:103). 

One  can  deduce  that: 

a.  Markers  were  relatively  consistent  in  focusing  more  on  surface  level  er- 
rors,  even  though  they  did  not  actively  work  together  and  even  though 
they  focused  on  different  surface  level  errors. 

b.  Errors  other  than  grammar,  spelling  and  punctuation  are  markedly  more 
difficult  to  identify. 

c.  Surface  structure  errors  bother  markers  - to  such  an  extent  that  markers 
may  even  ignore  other  errors.  Admittedly,  there  are  so  many  errors  in  some 
of  the  sample  essays  that  it  gets  extremely  difficult  to  mark  for  argument. 
See  example  texts  one  and  two  in  Addendum  B. 

d.  Writing  seems  to  be  an  effective  way  to  notice  a poor  lexicon. 

Marker  personal  favourites 

For  purposes  of  comparison,  the  inter-marker  consistency  (top  ten  used  tags)  are 
included  in  Addendum  D.  In  Table  2 the  tags  which  occurred  in  all  markers'  top 
ten,  are  presented.  A count  of  four  therefore  indicates  that  all  four  markers  used 
the  tag  as  one  of  their  most  frequently  used  tags,  while  a count  of  three  indicates 
that  three  of  the  four  markers  used  the  tag  as  one  of  their  most  frequently  used 
tags. 


Table  2:  Tags  that  occurred  in  makers’  personal  favourites 
Error  tag  Occurrence  in  top  ten  of  marker  favourites 


Punctuation  missing 
Spelling/typing  error 
Wrong  word 
Superfluous  general 
Word  form  wrong 
Better  word 
Capitalization 
Concord 

Omission  general 
Punctuation  wrong 


Tense 


2 
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There  is  very  low  inter-marker  consistency  evident  here.  Although  it  can  be  ar- 
gued  that  all  markers  should  have  rnarked  the  exact  same  essays,  the  focus  of  the 
experiment  was  not  just  on  consistency,  but  also  on  ease  of  use  for  the  marker.  A 
broader  spectrum  of  essays  to  mark  generated  a broader  spectrum  of  possible  uses 
of  the  tag  set  and  hence  a more  thorough  test  of  the  marking  system.  In  addition, 
these  essays  were  all  written  by  students  of  more  or  less  the  same  competency,  so 
the  comparison  (although  not  perfect)  can  still  be  seen  as  legitimate. 

There  are  only  three  tags  that  occur  in  all  four  markers'  top  ten.  These  are 
"punctuation  missing",  "spelling/typing  error  " and  "wrong  word".  The  only  tag 
to  occur  in  three  of  the  four  top  ten  lists  is  "superfluous  general"  which  is  used  to 
indicate  superfluous  words. 

If  markers  have  to  constantly  focus  on  incorrect  word  choice,  punctuation, 
overuse  of  words  and  incorrect  spelling,  it  indicate  two  possibly  interacting  issues: 

1.  Students  have  a very  poor  ability  to  make  themselves  understood,  which 
forces  markers  to  indicate  these  errors  in  an  attempt  to  point  out  that  they 
were  unable  to  understand  the  text. 

2.  Markers  are  overly  finical  with  regard  to  surface  level  errors  or  simply  find 
it  easier  to  comment  on  those. 

Leastused  tags 

Table  3 presents  the  tags  least  used  by  the  markers.  Although  these  tags  were 
available,  markers  tended  not  to  use  them  a lot. 

Regarding  the  least  used  tags,  the  following  issues  appear  to  be  significant: 

a.  Five  of  the  six  tags  in  the  dornain  "structure"  occur  in  the  least  used  tags. 
Markers  therefore  seldom  touched  upon  the  issue  of  paragraph  structure. 
The  domain  "structure"  falls  under  errors  of  discourse  and  refers  mainly 
to  errors  concerned  with  paragraphing. 

b.  Positive  comments  are  also  amongst  the  least  used  tags  with  only  two  of 
the  markers  ever  using  positive  comments  and  then  extremely  sparingly. 

c.  The  rest  of  the  least  used  tags  are  issues  of  grarnmar  that  are  either  low 
frequency  ("superfluous  tense  marker  ")  or  more  difficult  to  identify  ("as- 
pecterror"). 

The  results  indicate  that  the  markers  still  sometimes  have  editing  or  spell  check- 
ing  in  mind  when  marking  student  texts.  However,  one  should  keep  in  mind  the 
virtual  incomprehensibility  of  some  of  the  student  texts  (see  addendum  C).  With 
some  of  these  texts  it  would  be  very  difficult  to  comrnent  on  structure  since  it  is 
difficult  to  understand  the  text  in  the  first  place.  Most  of  the  texts  are  not  that 
difficult  to  comprehend  though,  so  in  spite  of  some  difficulties  the  question 
remains:  How  can  markers  be  assisted  to  be  more  than  spell  checkers  who  simply 
look  for  instances  of  surface  level  incorrectness? 
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Table  3:  Least  used  tags 


Subordinate 

Domain 

Set 

Knorm 

Mnorm 

Pnorm 

Tnorm 

Normed  Total 

Discourse 

Style 

Active  voice 

0 

0 

0 

1 

1 

Grammar 

Syntax 

Superfluous  tense  marker 

1 

0 

0 

0 

1 

Grammar 

Syntax 

Omission  tense  marker 

0 

0 

0 

1 

1 

Grammar 

Syntax 

Preposition  unnecessary 

0 

0 

0 

2 

2 

Discourse 

Style 

Passive  voice 

0 

0 

0 

2 

2 

Discourse 

Positive 

comments 

Good  reasoning 

2 

0 

0 

0 

2 

Grammar 

Morphology 

Wrong  form  - adjective 

2 

0 

0 

0 

2 

Grammar 

Syntax 

Negation  incorrect 

1 

0 

1 

0 

2 

Discourse 

Structure 

Paragraph  jumbled 

1 

1 

0 

0 

3 

Discourse 

Style 

Construction  overuse 

0 

1 

0 

2 

3 

Grammar 

Lexis 

Quantifier  error 

1 

0 

2 

0 

3 

Discourse 

Structure 

Paragraphing: 
relate  or  move 

0 

0 

3 

2 

5 

Discourse 

Style 

Gender  Bias 

0 

0 

0 

5 

5 

Grammar 

Lexis 

Wrong  time  / 
temporal  adverbial 

0 

3 

2 

0 

5 

Discourse 

Factual 

correctness 

Unsupported  argument 

0 

0 

0 

6 

6 

Grammar 

Lexis 

False  friend 

3 

0 

2 

1 

6 

Grammar 

Lexis 

Problem  with  conditional 

2 

0 

3 

1 

7 

Presentation 

Layout 

Layout  inhibits  reading 

0 

0 

7 

0 

7 

Discourse 

Structure 

Introduction  weak 

0 

0 

4 

3 

8 

Discourse 

Structure 

Paragraph:  Start  new 

3 

0 

3 

2 

8 

Grammar 

Morphology 

Wrong  form 
- past  participle 

2 

2 

2 

3 

9 

Discourse 

Style 

Register  too  formal 

8 

0 

0 

1 

9 

Discourse 

Positive 

comments 

Interesting  point 

8 

0 

2 

0 

10 

Grammar 

Morphology 

Aspect  error 

5 

0 

5 

0 

10 

Discourse 

Structure 

Paragraph:  weak 
opening  sentence 

3 

4 

0 

3 

10 

Grammar 

Syntax 

Preposition  unnecessary 

2 

4 

0 

5 

11 

Grammar 

Syntax 

Sentence  incomplete 

0 

1 

10 

1 

12 

Grammar 

Lexis 

Pronoun  wrong 

8 

0 

1 

6 

15 

Grammar 

Lexis 

Inappropriate  word 

13 

0 

0 

2 

15 

Discourse 

Style 

Verbosity 

1 

2 

2 

13 

18 
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Subordinate 

Domain 

Set 

Knorm 

Mnorm 

Pnorm 

Tnorm 

Normed  Total 

Discourse 

Factual 

correctness 

Opinion 

3 

2 

3 

11 

20 

Presentation 

Punctuation 

Apostrophe  error 

1 

12 

0 

7 

20 

Discourse 

Factual 

Reference  omitted  / 

0 

0 

18 

1 

20 

correctness 

wrong 

Grammar 

Syntax 

Unnecessary  pronoun 

7 

7 

0 

7 

21 

Discourse 

Factual 

correctness 

Unbalanced  statement 

7 

0 

1 

14 

22 

Grammar 

Lexis 

Wrong  modal 

4 

2 

2 

15 

23 

Grammar 

Morphology 

Wrong  form 
- present  participle 

8 

9 

5 

3 

25 

Grammar 

Syntax 

Omission  for  punctuation 

0 

15 

3 

9 

27 

Discourse 

Coherence 

Inconsistency 

12 

0 

9 

6 

27 

Close  analysis  ofthe  use  ofone  tag 

The  tag  "wrong  word"  was  selected  for  a closer  analysis  since  all  four  markers  had 
it  in  their  list  of  top  ten  used  tags.  The  results  indicate  that  intra-marker  consisten- 
cy  was  relatively  good,  while  inter-marker  consistency  needs  some  work.  Mark- 
ers  will  need  some  training  in  order  to  be  consistent  with  one  another,  or  will  need 
to  work  together  more  closely.  The  close  analysis  of  this  specific  feedback  tag 
highlighted  the  followingproblems: 

(i)  Favourite  generics:  Markers  tended  to  use  the  tag  as  a generic  term 
instead  of  using  more  specific  available  tags. 

(ii)  Doubles:  In  some  instances  more  than  one  tag  may  apply,  but  rnarkers 
reverted  to  tlae  one  they  have  used  previously. 

(iii)  Personal  preference  resulted  in  tags  of  "wrong  word"  where  "better  word" 
would  have  been  a better  option. 

(iv)  Incorrectly  tagged. 

(v)  Errors  that  were  difficult  to  classify. 

Each  of  these  will  be  discussed  briefly: 

(i)  Markers  tended  to  use  the  tag  as  generic  term  instead  of  using  more 
specific  available  tags. 

Example  1 : They  must  go  to  the  streets  to  beg  for  < Wrong  word  > 
money  to  eat  < /Wrong  word  > . 
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In  example  1,  the  tag"Sentence  Ambiguity"  could  have  worked  better.  "They"  do 
go  to  the  streets  to  beg  for  money  to  buyfood  to  eat.  The  problem  therefore  lies 
much  more  with  the  sentence  construction  than  with  the  word  choice. 

Example2: ...  an  <Wrong word>infinitive<AVrong word>  tapestry 
of  green  maize  and  yellow  sunflower 

In  example  2,  the  word  "infinitive"  should  be  "infinite".  A better  tag  would  there- 
fore  have  been  "word  form  wrong".  It  is  therefore  a morphological  error  rather 
than  a lexical  or  semantic  error.  On  the  other  hand,  one  can  make  an  argument  for 
the  tag  "better  word"  also  in  order  to  use  "seemingly  endless"  or  "never-ending77 
if  the  idea  of  an  "infinite"  farm  proves  problematic.  Part  of  the  problem  therefore 
lies  in  the  interpretation  of  the  error. 

The  problem  of  using  a favourite  tag  as  a generic  tag  can  be  overcome  with 
some  collaboration  between  the  markers  and  more  specialized  training  of  the 
markers.  Presumably  as  markers  get  more  used  to  the  system  and  get  to  know  the 
tags  better,  they  will  be  more  a ware  of  additional  tags  they  can  use  instead  of  their 
favourite  generic  tag.  The  process  of  providing  feedback  continuously  on  multi- 
ple  drafts  written  by  students,  could  also  assist  in  this. 

(ii)  Doubles:  More  than  one  possible  tag. 

Example3: . . . what  they  do  to  keep  <Wrong  word>this</Wrong 
word>  <Word  form  wrong>tradition/s<AVord  form  wrong>  from... 
In  example  3,  the  difficulty  lies  therein  that  there  are  multiple  possible  ways  to 
correct  this  sentence.  Should  itbe  "these  traditions"  or  "this  tradition"?  The  con- 
text  will  normally  dictate  the  answer.  In  this  specific  case,  however,  one  can  ask  if 
the  problem  is  a "wrong  word",  a "wrong  form"  or  an  "omission  plural  marker". 

Another  example  of  multiple  possible  interpretations  of  an  error  is  the  word 
"irregardless"  as  used  in  one  of  the  essays.  The  dictionary  classifies  it  as  informal 
so  it  is  a question  of  style  as  well  as  an  issue  of  "wrong  word".  The  tag  "better 
word"  could  work  as  well. 

A similar  occurrence  of  double  errors  is  with  run-on  sentences.  Do  you  classi- 
fy  it  as  "omission  punctuation"  or  "run-on  sentence"?  The  initial  idea  was  to 
simply  tag  double  errors  with  both  applicable  tags,  but  due  to  a technical  limita- 
tion  in  the  marking  program,  that  was  not  possible.  With  the  system  as  it  is  now, 
the  question  facing  the  marker  is  which  tag  to  use.  It  seems  markers  normally 
opted  for  their  "generic  favourite". 

(iii)  Personal  preferences 

Example  4:  In  this  essay  <Wrong  word>one</Wrong  word>  isgoing 
totry  to  <Wrong  word>prove</Wrong  word>  thatthe  prison  system 
is  outdated 
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In  example  4,  the  word  "prove"  could  also  be  tagged  as  "better  word"  if  the 
marker  was  of  the  opinion  that  "argue",  "show",  or  "demonstrate"  would  have 
been  a better  choice.  This  is,  however,  a harsh  judgement  by  the  marker  which  is 
clearly  indicative  of  a personal  preference. 

(iv)  Incorrectly  tagged 

Example  5: . . . how  the  Romans  whipped  and  < Wrong  word  > 
cruxivied</Wrong word>  them  ... 

The  word  "cruxivied"  in  example  5 is  definitely  not  a wrong  word.  It  is  the  correct 
word  that  had  been  spelled  incorrectly  so  the  tag  should  have  been  "spelling/ 
typing  error".  The  student  could  now  be  under  the  impression  that  "crucified"  is 
not  the  correct  word  to  use  and  will  struggle  in  vain  to  find  the  "correct"  word. 

Example  6:  Every  bank  has  different  options  regarding  a savings 
account  to  < Wrong  word  > consider.  </Wrong  word  > 

In  example  6,  the  word  "consider"  is  not  such  a big  problem  as  some  omitted 
words:  "Every  bank  has  different  options  regarding  a savings  account  one  has  to 
consider,"  is  a possible  correction. 

Example  7:  We  all  know  this  catchy  tune  and  I am  sure  a lot  of  us 
actuallygave  it  some  <Wrong  word>tough</Wrong  word>  one 
time  or  the  other. 

This  is  a spelling  error  in  example  7.  The  student  meant  "thought".  Because  of  the 
incorrect  tag  the  student  stHl  does  not  know  that  he/she  simply  misspelled  "thought" 
but  is  instead  under  the  impression  that  "thought"  is  the  wrong  word  to  use. 

(v)  Errors  that  were  difficult  to  classify 

Some  errors  were  difficult  to  classify  because  it  was  not  immediately  evident 
what  the  learner  wanted  to  say.  It  is  only  possible  to  tag  an  error  once  you  can 
establish  what  the  intended  meaning  was.  This  is  the  case  especially  when  learn- 
ers  write  very  long  "sentences"  without  any  verbs  or  punctuation.  Example  8 
illustrates  this  problem. 

Example  8:  Compulsory  modules  some  of  them  are  good  but  some  of 
the  they  are  full  of  nonsense  because  we  gain  nothing  from  them  while 
others  are  very  good  because  they  prepare  us  for  the  working 
enviroment  and  also  to  become  good  professionals.  Modules  like 
sociology  especially  if  you  want  to  do  community  work  is  great  and 
also  koms  because  we  learn  how  to  communicate  or  improve  our 
communication  skills  since  were  are  preparing  to  be  professionals. 
Modules  like  entr  and  wtll  I don't  really  know  why  we  should  do  them 
cause  according  to  me  is  total  waste  of  time  and  money.  Any  way  we 
just  have  to  do  them  because  we  don't  have  choice  at  all. 
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Consistency:  Marker  comments 

The  questions  asked  to  the  markers  brought  the  following  to  light: 

• Only  one  marker  was  able  to  correctly  identify  the  tag  he/she  used  most. 
The  others  indicated  error  tags  that  were  not  even  in  their  top  ten.  This 
indicates  that  the  markers  were  often  not  consciously  aware  of  what  they 
focused  on  even  though  they  were  under  the  impression  that  they  paid 
attention  to  more  than  "editing  errors".  However,  one  should  keep  in  mind 
that  three  of  the  four  markers  indicated  they  often  had  difficulty  in  under- 
standing  what  exactly  the  students  intended  to  say.  This  makes  it  almost 
understandable  that  they  focused  on  the  surface  level  errors  instead. 

• All  the  markers  indicated  that  they  found  the  students'  ability  to  present 
an  effective  argument  underdeveloped.  One  marker  explained,  "Some- 
times  students  showed  great  insight  and  had  impressive  ideas,  but  they 
were  unable  to  incorporate  them  into  the  argument.  Usually  any  insight 
was  lost  in  a sea  of  words." 

• On  the  other  hand,  one  marker  indicated  that  he/she  consciously  decided 
to  ignore  spelling  or  typing  errors  since  he/she  found  the  other  errors  of 
more  importance.  From  these  comments,  it  seems  that  the  marking  system 
should  also  have  been  tested  on  texts  other  than  the  Tswana  Learner  Cor- 
pus.  One  may  speculate  that  if  the  markers  are  able  to  understand  the  text 
better,  they  will  mark  it  with  more  care  and  more  comprehensively.  This 
will  hopefully  also  result  in  greater  consistency. 

• As  far  as  consistency  is  concerned,  all  the  markers  turned  out  to  be  more 
than  just  spell  checkers,  but  there  were  differences  in  what  they  focused 
on.  This  problem  can  probably  be  surmounted  if  the  system  is  used  as  part 
of  a well  structured  writing  lesson  where  all  the  markers  know  what  the 
aim  of  the  exercise  is,  and  therefore  focused  on  the  same  issues  in  unison. 

Ease  ofusefor  the  marker 

An  integral  part  of  the  project  to  provide  better  feedback  was  to  try  and  make  it 
easier  and  faster  for  the  marker  to  provide  more  (and  more  thorough)  feedback. 
Unfortunately  the  software  used  for  this  experiment  was  a prototype  version  and 
contained  many  bugs  and  system  limitations  which  hampered  the  process. 

Despite  the  bugs,  markers  indicated  that  they  could  get  a turn  of  speed  from 
the  system,  especially  once  they  got  to  know  the  tag  set.  Unfortunately,  it  still  took 
between  10  and  30  minutes  per  500  word  essay.  Before  any  rash  judgements  on  the 
time  effectiveness  of  the  system  can  be  made,  it  has  to  be  compared  to  normal 
manual  marking.  Even  if  it  tums  out  that  it  takes  just  as  long  to  mark  an  essay  with 
the  marking  system  as  with  traditional  manual  marking,  the  system  has  more 
advantages  than  traditional  marking. 
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It  seems  that  the  main  reasons  for  the  slow  marking  are  the  following: 

1.  Bugs  and  limitations  in  the  system. 

2 It  takes  a while  to  classify  an  error;  a simplified  tag  set  (less  elaborate)  may 
streamline  the  marking.  A balance  needs  to  be  found  between  the  explicit- 
ness  of  the  feedback  and  the  value  students  get  from  it. 

3.  The  markers  were  not  used  to  read  text  on  a computer  screen,  slowing 
their  reading  speed. 

Solving  these  problems  will  speed  up  the  marking.  In  addition,  the  following 
plug-ins  are  being  considered  for  the  system,  which  could  greatly  assist  markers 
in  speeding  up  their  marking  process: 

1.  A custom  spelling  and  grammar  checker  that  can  identify  and  mark  sur- 
face  errors  before  the  teacher  even  gets  the  text. 

2 A "focus"  function  which  only  allows  the  teacher  to  use  specific  tags  ena- 
bling  him/her  to  focus  on  certain  aspects  at  a time. 

3.  A teacher  prompt  function  reminding  the  teacher  to  use  a greater  variety 
oftags. 

4.  A voice  prompt  enabling  markers  to  use  their  voice  to  insert  an  error  tag 
instead  of  clicking  with  their  mouse. 

Unfortunately,  these  possible  solutions  are  time-consuming  and  expensive  to 
develop,  test  and  implement. 


6.  Conclusion 

This  paper  commenced  with  the  questions  of  whether  standardised  feedback 
could  be  used  consistently  and  if  it  would  make  the  life  of  the  marker  easier.  The 
answer  is  that  although  there  is  still  plenty  of  work  to  do,  the  initial  findings 
provide  plenty  of  positives.  The  system  still  needs  a lot  of  refinement. 

The  first  testing  of  the  system  indicated  that  it  has  advantages: 

1.  It  can  assist  markers  in  raising  their  awareness  of  errors. 

2 A regular  analysis  of  the  used  error  tags  could  assist  the  teacher  in  identi- 
fying  where  students  lack  knowledge. 

3.  A regular  analysis  of  the  error  tags  used  could  assist  the  teachers  in  identi- 
fying  where  they  are  overly  sensitive  to  a specific  error,  or  fail  to  pay  atten- 
tion  to  important  errors. 

In  addition,  the  experiment  emphasized  the  following  problems: 

1.  It  is  difficult  to  provide  effective  feedback  since  it  entails  tiring  thought 
processes  and  error  analysis. 
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2.  Detailed  feedback  on  surface  levels  errors  is  possible  in  a standardised 
way,  but  markers  will  need  some  assistance  to  consciously  move  away 
from  merely  editing  students'  work. 

3.  Some  errors  can  be  classified  in  more  than  one  way.  This  makes  it  difficult 
for  markers  to  be  consistent  in  how  they  mark.  The  problem  may  be  solved 
with  adaptations  to  the  error  tag  set. 

4.  At  present,  feedback  remains  a time-consuming  activity. 

5.  One  standardised  set  of  feedback  tags  does  not  seem  to  be  useful  for  differ- 
ent  levels  of  students,  since  the  weaker  students  make  so  many  surface 
level  errors  that  the  text  is  difficult  to  mark. 

Addressing  these  problems  will  still  require  lengthy  research  and  plenty  of  com- 
puter  programming,  but  at  least  a start  has  been  made  and  the  data  shows  it  to  be 
a start  in  the  right  direction. 
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Addendum  A:  Extract  from  tag  set 

Please  note  that  the  full  tag  set  could  not  be  included  due  to  space  constraints. 


Superordinate 

Domain 

Tag 

Feedback 

Example/explanation 

Xml  label 

Grammar 

Lexis 

Repetition 

You  use  the  same 
words  repeatedly. 

Find  different  words 
that  may  convey 
your  message  more 
clearly. 

Use  this  tag  when  you  realise  that  a student 
keeps  on  using  the  same  word.  E.g.:  If  a student 
use  the  word  “Good”  to  mean  “excellent"  and 
“strong"  and  “hard"  and  “pretty"  etc.  This  will  be 
context  sensitive.  If  you  have  to  use  the 
“Word:  better  word"  tag  a lot  for  the  same  word, 
rather  start  using  the  “Word:  repetition"  tag. 

GLRE 

Grammar 

Lexis 

Word 

choice 

obscuring 

meaning 

This  word  is  not  clear 
enough.  Find  a better 
word  to  say  what  you 
want  to  say. 

Use  this  when  another  word  would  make  the 
intended  meaning  much  clearer.  E.g.:  “Only  third 
year  students  were  able  / allowed  to  go."  All 
were  able  to  go,  but  all  were  not  allowed  to  go. 

GLWC 

Grammar 

Lexis 

Word 

form 

wrong 

This  word  should 

have  been  in  a 

different  form 

for  this  context. 

Use  this  for  words  in  the  wrong  form,  not  covered 
by  the  other  labels  below. 

GLWF 

Discourse 

Style 

Active 

voice 

The  passive  voice 
might  be  more 
appropriate  here. 

Use  this  where  the  student  used  the  active  voice, 
but  in  your  opinion,  the  passive  voice  would  have 
worked  better. 

DSAV 

Grammar 

Morphology 

Wrong 

form 

- present 
participle 

The  wrong  form 
of  the  word.  Use 
the  “-ing"  form 
of  the  word. 

1 am  busy  work  in  the  garden. 

GMWO 

Grammar 

Morphology 

Wrong 
form  - 
adjective 

Wrong  word  form. 

Use  the  correct 

form  of  the  word. 

Use  this  when  the  learner  should  have  used  the 
adjective  form  of  the  word,  e.g.  “He  gave  me 
rot  apples." 

GMWF 

Grammar 

Lexis 

Wrong 

Time  / 

temporal 

adverbial 

This  time-word 

does  not  fit  the 
rest  of  the  essay. 

Use  this  when  a student  uses  e.g.  a word  in  the 
past-tense  when  the  whole  essay  is  written  in  the 
present  tense. 

GLWT 

Grammar 

Lexis 

Wrong 

word 

This  is  the  wrong 
word.  Find  and 

use  the  correct 

word  for  the  context. 

Use  this  when  a student  should  have  used  another 
word  instead  e.g.  “Students  should  be  leamed 
(taught)  to  ..."  or  “injury  (damage)  to  property." 
Property  cannot  hurt. 

GLWW 
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Addendum  B:  Questions  to  markers  who  used  the  marking  system 

Please  answer  the  following  questions  regarding  your  experience  using  the  mark- 

ing  system  last  year.  You  may  answer  in  the  document  and  just  email  it  back. 

Don't  be  shy  to  make  positive  or  negative  comments. 

In  the  questions,  I distinguish  between: 

A)  System:  the  computer  program. 

B)  Error  tags:  the  error  categories  (the  buttons  you  used  and  the  list  of  catego- 
ries  you  had.) 

1 What  is  your  definition  of  "error  "? 

2 Which  error  tag  do  you  think  you  used  the  most? 

3 Which  errors  were  the  most  difficult  to  identify? 

4 Were  you  pressed  for  time  when  marking  these  essays? 

5 How  much  longer  do  you  think  it  took  you  to  mark  an  essay  with  the  system 
than  without  it? 

6 Can  you  read  a text  and  ignore  (or  fail  to  notice)  spelling  errors  or  grammat- 
ical  errors? 

7 What  was  your  overall  impression  of  the  quality  of  the  students'  writing? 

8 What  is  your  overall  impression  of  the  students'  ability  to  present  an  effec- 
tive  and  clear  argurnent  in  their  texts? 

9 Do  you  read  for  spelling  and  grammar  errors  separately  from  reading  for  the 
argument  in  a text  or  do  you  pay  attention  to  both  at  the  same  time? 

10  Were  there  any  error  tags  in  the  system  you  did  not  understand  or  did  not 
know  how  to  use? 

11  Do  you  think  the  tags  available  in  the  system  raised  your  awareness  of  possi- 
ble  errors?  If  so,  please  give  an  example. 

12  What  were  the  most  common  errors  students  made  in  their  writing? 

13  How  often  would  you  say  was  it  dif  ficult  to  decide  which  tag  to  use?  Give  an 
example  if  you  can. 

14  How  often  did  you  use  the  system?  Did  you  use  it  regularly  or  now  and  then 
for  abigbatch. 

15  Did  it  get  easier  to  use  the  system  after  using  it  for  a while? 

16  What  are  your  recommendations  to  improve  the  error  tag  set? 

17  What  are  your  recornmendations  to  improve  the  marking  system? 

18  Any  other  comments  you  wish  to  make? 
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Addendum  C:  Examples  ofstudent  writing 

1.  Compulsory  modules  some  of  them  are  good  but  some  of  the  they  are  full  of 
nonsense  because  we  gain  nothing  from  them  while  others  are  very  good 
because  they  prepare  us  for  the  working  enviroment  and  also  to  become  good 
professionals.  Modules  like  sodology  especially  if  you  want  to  do  community 
work  is  great  and  also  koms  because  we  learn  how  to  communicate  or  im- 
prove  our  communication  skills  since  were  are  preparing  to  be  professionals. 
Modules  like  entr  and  wtll  I don't  really  know  why  we  should  do  them  cause 
according  to  me  is  total  waste  of  time  and  money.  Any  way  we  just  have  to  do 
them  because  we  don't  have  choice  at  all. 

2.  Poverty  is  short  fall  of  consumption  or  income  if  somebody  can  not  meet  the 
basic  needs  he  or  she  is  regarded  as  a poor-man.  It  has  Found  that  African 
countries  are  under  developed  so  is  where  the  poverty  is  highly  located.As 
poverty  is  highly  concentrated  in  rural  area,  or  town  outskirt  and  women  and 
children  and  tenagers.  Aids  goes  hand  in  hand  with  poverty  because  those 
women  not  working  had  a lot  of  children  and  they  are  straving.  So  women  as 
parents  has  to  find  food  a clothing  and  school  for  their  children  therefore  the 
only  altarnative  is  to  practice  prostitution  or  forced  to  be  married  by  those 
who  can  help  them.  In  this  way  teenagers  would  go  out  with  elders  especica- 
ly  businessmen  to  facilitates  funds. 

3.  Our  South  African  players  are  not  paid  well  they  are  being  underpaid  they 
dont  get  the  salaries  that  befit  their  job  our  players  are  putting  the  country  in 
to  high  places  they  are  proving  it  to  the  world  that  they  can  compete  with 
other  strong  countries  Our  officials  must  start  thinking  properly  the  high 
salaries  that  are  being  paid  to  the  officials  who  are  doing  nothing  just  sitting 
the  whole  day  in  their  offices  and  attending  alot  of  meetings  making  business 
contacts  for  them  selfs  the  real  heroes  are  being  underpayed  the  reason  for  our 
players  to  íeave  the  country  to  go  and  play  in  foreign  countries  is  that  they  are 
paid  well  they  get  the  money  that  they  are  playing  for  those  the  reason  when 
our  players  are  in  foreign  countries  they  play  the  sport  with  pride  in  them 
they  get  proper  treatment  our  players  must  be  paid  correctly  because  the  sport 
that  they  are  playing  is  their  career  they  have  families  to  feed  they  are  not  only 
in  the  sport  for  the  sake  of  money  but  because  of  the  love  for  the  sport  they  are 
proffessionals  they  master  the  sport  that  they  are  playing  it  is  of  no  use  repre- 
senting  your  country  but  you  are  not  paid  according  to  the  job  that  you  are 
doing  if  you  do  a job  correctly  you  expect  to  be  rewarded  accordingly  so  our 
players  are  our  pride  and  they  are  putting  the  country  into  greater  heights  so 
if  they  can  start  getting  decent  salaries  than  there  wont  be  a need  for  them 
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leaving  the  country;  The  reason  why  they  are  leaving  the  country  is  because 
they  are  offered  better  opportunities  they  make  alot  of  money  in  a short  space 
of  time  and  they  get  alot  of  expoture  when  they  are  playing  in  foreign  coun- 
tries;  If  they  can  be  paid  as  the  foreign  countries  are  paying  them  than  there 
wont  be  a need  for  them  to  leave  the  country  given  the  same  opportunities  as 
those  given  to  them  by  the  foreign  countries;  So  our  officials  must  start  think- 
ing  properly  and  try  to  improve  the  way  our  players  are  treated  by  giving 
them  the  correct  salaries  or  else  our  country  will  endup  with  no  players  all  the 
player  will  leave  for  European  countries  the  europeans  will  take  all  the  good 
players  and  at  the  end  the  country  will  be  left  with  nothing  it  will  be  unable 
to  compete  with  their  counterpaths  because  all  the  good  player  will  be  play- 
ing  for  European  countries  so  let  us  start  taking  this  thing  into  consoda  ration 
and  pay  our  player  more  so  that  they  can  stay  at  home  and  make  us  proud  of 
them.  After  all  "Home  brewed  is  best". 
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Addendum  D 


KK  normed 
number  of 


Subordinate 

Domain 

Tag 

occurrences 

1 

Presentation 

Spelling 

Spelling/typing  error 

105 

2 

Grammar 

Lexis 

Wrong  word 

92 

3 

Discourse 

Coherence 

Reasoning  inconclusive 

73 

4 

Discourse 

Style 

Sentence  vague 

53 

4 

Grammar 

Syntax 

Superfluous  general 

53 

6 

Discourse 

Cohesion 

Sentence  cohesion 

46 

7 

Grammar 

Morphology 

Tense 

42 

8 

Grammar 

Syntax 

Omission  general 

40 

9 

Presentation 

Punctuation 

Punctuation  wrong 

25 

10 

Discourse 

Coherence 

Relevance  to  topic 

23 

10 

Presentation 

Punctuation 

Punctuation  missing 

23 

K-  PERSONAL  FAVOURITES 

Subordinate 

Domain 

Tag 

MB  normed 

number  of 

occurrences 

1 

Presentation 

Capitalization 

Capitalization 

130 

2 

Presentation 

Spelling 

Spelling/typing  error 

92 

3 

Grammar 

Lexis 

Article  missing 

62 

4 

Grammar 

Morphology 

Omission  plural  marker 

61 

5 

Grammar 

Morphology 

Concord 

46 

6 

Presentation 

Punctuation 

Punctuation  missing 

44 

7 

Grammar 

Lexis 

Wrong  word 

40 

8 

Grammar 

Lexis 

Determiner  Incorrect 

39 

9 

Grammar 

Lexis 

Word  form  wrong 

36 

10 

Grammar 

Syntax 

Superfluous  general 

35 

MB  PERSONAL  FAVOURITES 
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P normed 
number  of 


Subordinate 

Domain 

Tag 

occurrences 

1 

Presentation 

Spelling 

Spelling/typing  error 

150 

2 

Grammar 

Lexis 

Wrong  word 

87 

3 

Grammar 

Lexis 

Better  word 

81 

4 

Grammar 

Morphology 

Concord 

79 

5 

Presentation 

Layout 

Layout  error 

57 

6 

Presentation 

Punctuation 

Punctuation  missing 

47 

7 

Grammar 

Lexis 

Word  form  wrong 

40 

8 

Grammar 

Lexis 

Preposition  wrong 

35 

9 

Presentation 

Capitalization 

Capitalization 

33 

10 

Presentation 

Punctuation 

Punctuation  wrong 

28 

P PERSONAL  FAVOURITES 

Subordinate 

Domain 

Tag 

T normed 

number  of 

occurrences 

1 

Presentation 

Spelling 

Spelling/typing  error 

62 

2 

Grammar 

Syntax 

Superfluous  general 

59 

3 

Presentation 

Punctuation 

Punctuation  missing 

51 

4 

Grammar 

Lexis 

Wrong  word 

47 

5 

Discourse 

Factual  correctness 

Facts  wrong 

44 

6 

Discourse 

Style 

Register  too  informal 

42 

7 

Grammar 

Morphology 

Tense 

38 

8 

Grammar 

Lexis 

Better  word 

34 

9 

Discourse 

Cohesion 

Reference  vague 

33 

10 

Grammar 

Syntax 

Omission  general 

32 

T PERSONAL  FAVOURITES 

Looking  into  the  seeds  of  time:  Developingacademic  literacy  in 
high  poverty  schools 

Elizabeth  ] Pretorius 
University  of  South  Africa 


Ifyou  can  look  into  the  seeds  oftime 

And  say  which  seeds  will  grow  and  which  will  not, 

Speak  then  to  me  . . . (Macbeth  Act  I,  Scene  iii) 


Looking  into  the  seeds  of  time:  Developing  academic  literacy  in  high  poverty 
schools 

Much  ofthe  literature  on  the  assessment,  development  and  redress  ofacademic 
literacy  comes  from  the  higher  education  sector.  In  contrast,  this  article  turns  an 
investigative  gaze  on  academic  literacy  in  three  primary  schools  in  a disadvan- 
taged  area,  and  compares  the  language  and  reading  accomplishments  ofGrade  7 
learners  in  theirhome  language,  Northern  Sotho,  and  in  English,  the  language  of 
leamingand  teaching.  On  the  basis  of  these  findings  implications  are  drawn  about 
the  development  ofacademic  literacy  in  primary  schools  and  the  dynamics  of 
student  unpreparedness  within  thebroader  educational  system.  In  conclusion  it  is 
argued  that  current  learning  and  teaching conditions  in  South  Africa  afford  a 
rather  sombre  view  ofthe  development  ofacademic  literacy  in  primary  schools. 


1 . Introduction 

In  one  of  the  early  scenes  of  the  play  Macbeth,  Banquo  and  Macbeth,  returning 
from  battle,  are  met  by  three  witches  who  prophesise  (in  ambiguous  phrases,  as 
witches  are  wont  to  do)  what  will  happen  to  Macbeth.  Curious  about  his  own 
future,  Banquo  asks  the  witches  to  "look  into  the  seeds  of  time/And  say  which 
seeds  will  grow  and  which  will  not"  in  his  situation.  While  it  was  common 
during  the  Middle  Ages  for  witches  to  foretell  the  future,  in  today's  world  we 
often  look  to  scientific  research  for  guidance  as  to  which  factors  might  predict 
specific  behaviours  or  events. 

This  study  examines  the  reading  accomplishment  of  Grade  7 learners  in  three 
different  schools  in  an  urban  South  African  township  to  see  what  this  reveals 
about  the  development  of  academic  literacy  in  primary  schools,  and  to  consider 
what  light  the  findings  might  shed  on  the  phenomenon  of  student  unprepared- 
ness  currently  being  experienced  in  the  higher  education  sector  in  South  Africa. 
While  this  study  cannot  claim  the  same  accuracy  of  prediction  as  that  of  the 
witches  in  Macbeth,  some  tentative  predictions  are  made  from  the  findings  as  to 
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which  academic  seeds  sown  in  primary  school  "are  likely  to  grow  and  which 
will  not". 

Background. 

In  South  Africa  concern  is  being  expressed  over  the  academic  preparedness  of 
students  entering  tertiary  level.  Over  the  years,  Grade  12  school  leavers'  results 
have  become  progressively  variable.  Although  school  exit  scores  in  the  top  ranges 
are  strongpredictors  of  performance  at  tertiary  level,  predictability  becomes  ten- 
uous  with  middle  and  lower  range  scores.  In  2005  Higher  Education  South 
Africa  commissioned  the  implementation  of  the  National  Benchmark  Test 
Project1  whose  brief,  basically,  is  to  "assess  students'  levels  of  academic 
readiness  ...  prior  to  possible entry  to  higher  education"  (Griesel, 2006). 
Three  aspects  of  academic  readiness  are  covered  by  the  set  of  national 
benchmark  tests,  viz.  academic  literacy,  quantitative  literacy  and  mathe- 
matics. 

Numerous  scholars  have  researched  the  knowledge  and  skills  that  char- 
acterise  academic  literacy  (e.g.  Corson,  1997;  Cummins,  1979, 1991, 2000; 
Thomas  & Collier  1997;  Bachman  & Palmer  1996 2).  These  include  factors 
such  as  vocabulary,  grammatical  insight,  inferencing,  distinguishing  main 
from  secondary  ideas,  perceiving  relations  between  text  entities,  knowl- 
edge  of  genre  structure  and  conventions,  understanding  visual  informa- 
tion,  sequencing,  and  so  on.  A combination  of  these  abilities  enables  stu- 
dents  to  maximise  learning  opportunities  and  read  to  learn  effectively.  Mayer 
(1992)  argues  that  academic  learning  is  underpinned  by  the  ability,  inter  alia,  to 
"focus  attention  on  relevant  information,  to  build  connections  among  the  rele- 
vant  pieces  of  information,  and  to  build  connections  between  old  and  new  knowl- 
edge"  (1992:256).  Over  and  above  these  cognitive-linguistic  skills  and  knowledge 
bases,  academic  literacy  also  entails  membership  of  a particular  group  and  orien- 
tation  towards  a set  of  behaviours  and  practices  evinced  by  the  group  (Gee,  1996). 

Even  though  academic  literacy  is  most  often  associated  with  secondary  and 
tertiary  education,  academic  literacy  has  its  roots  in  primary  school.  Yet  research 
into  academic  literacy  at  this  level  tends  to  be  an  under-researched  domain  with- 
in  the  South  African  context.  In  this  article  the  relationship  between  reading, 
academic  performance  and  academic  literacy  is  briefly  examined.  The  study  is 
then  described  and  the  results  of  Grade  7 reading  comprehension  tests  from 
different  learning  contexts  within  the  same  township  are  compared  and  dis- 
cussed.  Finally,  the  implications  of  these  findings  for  the  development  of  aca- 
demic  literacy  in  South  African  schools  are  considered. 


1 The  Centre  for  Higher 
Education  Development  at 
the  University  of  Cape  Town 
has  been  tasked  with  the 
managementand  develop- 
ment  of  this  project. 


2 The  academic  literacy  test  of 
the  NBTP  uses  as  its  basic 
framework  the  specifications 
identified  by  Bachman  & 
Palmer  19%. 
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2.  Reading,  academic  literacy  and  academic  performance 

In  2001  the  Department  of  Education  undertook  its  first  large  scale  systemic  eval- 
uation  of  reading  and  writing  in  Grade  3 across  all  nine  provinces.  The  results 
showed  a mean  of  38%  in  the  home  language  in  Grade  3 (Department  of  Educa- 
tion,  2003).  In  2005  the  results  of  the  systemic  assessment  of  reading  and  writing  in 
Grade  6 were  released.  Here  too  the  results  showed  a national  mean  of  38%  in  the 
language  of  learning  and  teaching  (LoLT):  63%  of  learners  were  found  to  be 
performing  in  the  'Not  Achieved'  band  (Department  of  Education,  2005).  Clearly, 
these  results  indicate  low  literacy  accomplishments,  yet  it  is  on  this  basis  that  the 
subsequent  development  of  academic  literacy  is  supposed  to  be  founded. 

'General ' language  proficiency  versus  academic  language  proficiency 
Language  development  during  the  school  years  becomes  increasingly  differenti- 
ated  according  to  particular  demands  and  contexts,  each  of  which  have  their 
own  specific  functions,  linguistic  registers  and  conventions.  The  differences  be- 
tween  spoken  and  written  modes  of  communication  were  first  articulated  by 
Bemstein  in  the  1960s  when  he  distinguished  between  restricted  and  elaborated 
codes  of  language  use  (Bernstein,  1966).  Cummins  later  proposed  a distinction 
between  two  kinds  of  language  proficiency,  based  on  the  context  and  the  func- 
tions  that  it  serves,  namely  Basic  Interpersonal  Communicative  Skills  (BICS)  and 
Cognitive  Academic  Language  Proficiency  (CALP)  (Cummins,  1979).  Later,  Cum- 
mins  emphasised  the  intersection  of  two  continua,  one  relating  to  the  cognitive 
demands  of  the  task  and  the  other  to  the  extent  of  it  occurring  in  embedded  or 
reduced  contexts  (Cummins,  1991, 2000). 

BICS,  used  in  everyday  communication,  is  more  context-embedded  in  that  it 
contains  paralinguistic  features  (gestures,  intonation,  facial  expression,  etc)  to 
convey  meaning  and  facilitate  comprehension,  as  well  as  deictic  items  whose 
meaning  can  be  recovered  from  the  interactional  context.  Academic  language 
proficiency,  on  the  other  hand,  involves  use  of  a more  context  reduced  language 
associated  with  written  language  and  with  the  more  formal  aspects  of  classroom 
and  lecture-hall  language  use  typical  of  the  learning  context.  This  is  not  to  say 
that  written  language  is  context-free,  rather  that  meaning  is  located  in  the  text  to 
a larger  extent  than  is  the  case  in  oral  discourse.  It  is  the  ability  to  access,  under- 
stand  and  convey  information  from  and  into  written  language  rather  than  oral 
language  that  accounts  for  success  in  the  learning  context. 

Crossing  the  bridgefrom  oral  to  written  language 

Academic  skills  in  reading  and  writing  take  longer  to  develop  than  speaking  and 
listening  skills.  All  children  acquire  BICS  in  their  primary  language  by  the  time 
they  start  school.  They  start  acquiring  academic  language  when  they  learn  to 
read  and  write  and  are  exposed  to  written  forms  of  language.  Hakuta  et  al.'s  study 
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(2000,  in  Cummins,  2000:58)  showed  that  oral  proficiency  in  an  additional  lan- 
guage  takes  3-5  years  to  develop  whereas  academic  profidency  can  take  4—7  years 
to  develop.  In  addition,  the  processes  and  skills  involved  in  the  development  of 
academic  literacy  do  not  all  develop  at  the  same  time  and  are  largely  dependent 
on  the  course  of  reading  development. 

Automaticity  in  reading 

Automaticity  refers  to  the  rate,  fluency  and  accuracy  with  which  readers  recog- 
nise  words  and  place  the  information  in  short  term  memory  (STM)  without  hav- 
ing  to  'sound  out'  words.  A lack  of  automatidty  in  decoding  can  interfere  with 
comprehension  processes  and  make  reading  effortful.  As  Kucer  (2005:107)  ex- 
plains,  "STM  becomes  overwhelmed  with  bits  and  pieces  of  discourse.  The  read- 
er  is  unable  to  make  sense  of  the  contents  in  STM  because  not  enough  informa- 
tion  is  available".  Automatidty  develops  through  constant  exposure  to  meaning- 
ful  texts.  From  Grades  1 to  3 the  technical  aspects  of  reading  (decoding)  are  em- 
phasised.  By  the  end  of  Grade  3 decoding  skills  should  be  well  developed  and 
become  increasingly  automatised.  Grades  4—7  are  crudal  for  the  further  develop- 
ment  of  automatic  decoding  as  well  as  strong  comprehension  reading  abilities  for 
it  is  on  these  foundational  abilities  that  academic  literacy  is  developed. 

Text  demands 

From  Grade  4 onwards  literacy  becomes  a vehicle  for  transmitting  information 
via  informational  and  expository  texts.  Academic  language  profidency  demands 
increase  significantly  as  texts  start  taking  on  forms  and  functions  not  previously 
encountered.  Expository  texts  contain  less  familiar  words  not  encountered  in 
everyday  language,  they  contain  longer  and  more  complex  sentences,  conceptu- 
ally  the  texts  become  more  dense,  complex  and  abstract.  Visual  literacy  demands 
also  increase  as  learners  'read'  increasingly  sophisticated  tables,  charts,  maps  and 
diagrams.  Cause  and  effect,  compare  and  contrast,  and  problem-solution  struc- 
tures  dominate.  Even  learners  who  have  mastered  the  basics  of  storybook  reading 
may  experience  problems  when  they  encounter  extended  texts,  new  registers  and 
words. 

Comprehension  processes 

Comprehension  involves  not  only  the  processing  of  linguistic  information  but 
also  the  processing  of  textual  knowledge  and  general  background  knowledge, 
which  entail  massive  amounts  of  cognitive  processing  involving  inferring,  un- 
derstanding,  integrating,  evaluatinginformation  within  and  across  texts,  recog- 
nising  inconsistencies  in  text  information,  monitoring  the  comprehension  proc- 
ess  and  applyingrepair  strategies  when  comprehension  breaks  down.  Compre- 
hension  enables  the  addition  of  new  knowledge  gained  from  texts  to  existing 
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knowledge  bases  in  memory,  and  the  modification  of  existing  knowledge  bases 
in  memory  in  response  to  information  acquired  from  texts. 

Ironically,  after  the  Foundation  phase  very  little  attention  is  given  to  explicitly 
teaching  reading  even  though  learners  are  increasingly  expected  to  read  to  learn 
from  content  subject  textbooks,  putting  new  linguistic  and  cognitive  demands 
on  learners.  As  Kucer  (2005: 34)  points  out,  "things  that  could  be  overlooked  or 
avoided  in  shorter  'single  sitting'  texts  become  increasingly  problematic".  In  many 
South  African  classrooms  teachers  tend  to  assume  that  the  literacy  basics  have 
been  taught  and  so  they  concentrate  on  content  subject  information.  Yet,  for 
many  leamers  the  transition  from  decoding  to  comprehension  does  not  happen 
easily,  especially  if  they  are  not  regularly  exposed  to  meaningful  reading  activi- 
ties  involving  extended  and  authentic  texts. 

Research  questions 

While  academic  literacy  involves  more  than  being  able  to  decode  and  compre- 
hend  texts,  the  knowledge,  skills  and  processes  involved  in  reading  form  the  core 
of  academic  literacy.  Without  automatic  decoding  and  sound  comprehension 
abilities,  academic  literacy  cannot  develop  properly. 

By  the  time  that  Grade  7s  reach  the  final  year  of  primary  school  they  are 
expected  to  have  developed  various  comprehension  competencies  involvingin- 
terpretation,  synthesis  and  evaluation.  In  reality  the  picture  may  be  very  differ- 
ent.  The  research  reported  on  in  this  article  addresses  the  question  of  academic 
literacy  by  examining  primary  school  exit  level  reading  abilities  in  both  the  Ll, 
Northem  Sotho  (henceforth  N Sotho),  and  English,  the  language  of  leaming  and 
teaching.  There  are  four  questions  that  inform  the  study: 

1.  What  is  the  reading  profile  of  Grade  7 leamers  from  three  different  urban 
township  schools,  in  N Sotho  and  English? 

2 How  does  reading  performance  in  Grade  7 relate  to  academic  perform- 
ance? 

3.  What  are  the  implications  of  these  findings  for  the  development  of  aca- 
demic  literacy  in  primary  schools? 

4 What  light  does  this  research  shed  on  the  future  preparedness  (or  not)  of 
students  entering  the  HE  sector? 


3.  Methodology 

Broader  context 

The  three  primary  schools  from  which  the  data  were  obtained  are  all  situated  in 
a predominantly  N Sotho/Tswana  speaking  township  in  Gauteng  province.  There 
is  one  private  and  26  state  primary  schools  in  the  township.  Of  the  state  primary 
schools,  10  are  predominantly  N Sotho  speaking,  9 Tswana,  3 Zulu,  2 Tsonga,  1 
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Venda  and  1 South  Sotho.  In  the  majority  of  these  schools,  initial  schooling  takes 
place  in  an  African  language,  from  Grade  1 to  Grade  3.  The  switch  to  English  as 
LoLT  is  made  in  Grade  4.  Thereafter  the  specific  African  language  is  taught  as  a 
first  language  subject. 

The  intervention  project 

Two  of  the  above  primary  schools  (Schools  B and  P)  are  involved  in  a reading 
intervention  programme,  the  aim  of  which  is  to  make  reading  an  integral  part  of 
daily  school  activities.  It  is  hoped  that  by  developing  a culture  of  reading  these 
schools  will  be  able  to  improve  the  overall  language  and  academic  development 
of  the  learners.  To  this  end  a multi-level  approach  has  been  adopted  that  empha- 
sises  the  building  up  of  print-  based  resources  as  well  as  capacity,  and  involves 
the  participation  of  the  learners,  teachers  and  parents. 

The  intervention  project  assists  the  schools  in  setting  up  a functional  school 
library  where  leamers  have  easy  access  to  age  appropriate  books  in  both  N Sotho 
and  English.  Besides  the  library,  the  schools'  resources  are  also  enhanced  by 
making  teachers  at  all  grade  levels  aware  of  the  need  to  create  print-rich  class- 
room  environments. 

Because  literacy  resources  have  no  value  if  not  used  properly,  teachers  (and 
parents)  need  to  be  shown  what  to  do  with  books.  The  intervention  thus  also 
focuses  on  developing  the  instructional  capacity  of  the  teachers  and  the  support- 
ive  capacity  of  the  parents.  Workshops  are  held  fortnightly  with  the  teachers  after 
school  to  increase  teachers'  understanding  of  the  reading  process.  Arrangements 
are  also  made  for  teachers  to  take  tums,  by  prior  appointment,  to  spend  a morn- 
ing  observing  good  reading  practice  in  a grade  equivalent  classroom  in  a highly 
effective  school  where  reading  is  a priority  (e.g.  School  M below). 

A family  literacy  component  is  also  included  in  the  project  to  involve  parents 
more  actively  in  the  literacy  development  of  their  children.  To  this  end  a series  of 
Family  Literacy  workshops  are  held  for  parents.  The  aim  of  these  workshops  is  to 
draw  parents'  attention  to  the  importance  of  reading,  to  encourage  them  to  read 
to  their  children  and/or  to  listen  to  their  children  reading,  to  take  an  interest  in 
children's  school  activities,  make  time  and  space  available  in  the  home  for  home- 
work,  encourage  membership  of  the  local  community  library,  and  so  on. 

Schools  B and  P (state  schools) 

School  B has  over  600  learners  and  a staff  of  16  teachers.  The  school  serves  a 
sodoeconomically  disadvantaged  community.  School  fees  were  R120  (about  $20) 
per  annum  but  at  the  end  of  2006  the  school  was  declared  a non-fee  paying 
school.  The  school  has  a feeding  scheme,  where  400  children  are  fed  once  a day. 
For  many  of  these  children,  this  is  the  only  meal  of  the  day. 

There  are  two  classes  at  each  grade  level.  In  the  Foundation  Phase,  there  are 


Ensovoort:  jaargang  1 1 , nommer  2,  2007 


1 1 1 


about  35  children  per  elass.  This  increases  to  around  50  per  class  in  Grade  7.  N 
Sotho  is  the  initial  language  of  leaming  and  teaching  from  Grade  R to  Grade  3, 
after  which  English  becomes  the  LoLT  N Sotho  is  taught  as  a subject  from  Grades 
4-7.  Although  many  children  come  from  homes  in  which  a variety  of  African 
languages  are  spoken,  about  70%  of  the  learners  at  this  school  come  from  prima- 
rily  N Sotho  speakinghomes. 

Similar  to  School  B,  School  P also  has  about  600  leamers  and  a staff  of  16 
teachers.  It  also  serves  a low  socioeconomic  community,  it  too  has  a school  feed- 
ing  scheme,  and  most  of  the  learners  at  this  school  also  come  from  primarily  N 
Sotho  speaking  homes.  However,  unlike  School  B where  initial  literacy  and  nu- 
meracy  is  taught  in  N Sotho  to  the  end  of  Grade  3,  School  P has  a 'straight  for 
English'  policy  from  Grade  1.  N Sotho  is  taught  as  a subject  from  Grades  2 to  7. 

For  ease  of  reference,  Schools  B and  P will  be  referred  to  collectively  as  the 
'township'  schools.  To  monitor  project  progress,  all  the  Grade  7 learners  at  both 
schools  are  tested  each  year  for  language  and  reading  ability  in  N Sotho  and 
English. 

School  M (private  school) 

School  M is  a small  private  primary  school  that  was  opened  in  the  township  in 
1991.  Even  though  it  serves  the  same  township  community,  many  of  the  children 
at  this  school  come  from  higher  socioeconomic  homes.  However,  there  are  also 
several  children  from  poor  homes  who  attend  the  school  on  scholarships. 

The  classrooms  are  well  resourced  and  the  teachers  well  qualified,  experi- 
enced  and  dedicated.  Classes  are  small  (abut  25  learner  per  class).  Reading  and 
storybooks  are  an  integral  part  of  each  classroom  in  the  lower  grades,  and  teach- 
ers  have  high  reading  expectations  of  learners.  Teachers  from  Schools  B and  P 
attend  the  private  school  for  classroom  observations  and  occasional  workshops, 
and  closer  ties  are  being  forged  between  these  schools. 

The  school  has  a 'straight  for  English'  policy.  The  learners  at  this  school  are 
not  linguistically  homogeneous  but  speak  different  African  languages  at  home. 
No  African  languages  are  taught  as  subjects.  Unlike  many  private  schools  or  ex- 
Model  C schools,  the  leamers  at  this  school  do  not  have  peers  for  whom  English 
is  an  Ll. 


4.  Language  and  reading  assessment 

The  2006  cohorts  of  Grade  7 learners  at  the  three  schools  were  administered  a 
reading  test  in  English  and  in  N Sotho.  The  learners  were  also  all  given  a lan- 
guage  test  in  English  and  N Sotho  to  enable  exploration  of  the  language-reading 
relationship.  Since  no  African  languages  are  taught  as  a subject  at  School  M,  the 
Grade  7s  at  this  school  only  completed  the  English  language  and  reading  tests. 
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Language  proficiency 

In  this  study  language  proficiency  was  operationalised  as  performance  on  a dic- 
tation  test  in  each  language.  According  to  Oller  (1979:58),  dictation  correlates  "at 
surprisingly  high  levels  with  a vast  array  of  other  language  tests".  This  correla- 
tion  points  to  dictation  tasks  tapping  into  similar  knowledge  sources  that  stand- 
ardised  language  tests  tap  into  but  it  does  so  via  the  auditory  rather  than  the 
written  medium. 

A dictation  test  is  an  integrative,  holistic  language  test  - provided  it  is  admin- 
istered  appropriately.  A passage  that  is  dictated  word-for-word  becomes  a short 
term  memory  test  and  hence  not  much  use  as  a measure  of  language  knowledge. 
Instead,  the  dictation  passage  is  first  read  at  normal  conversational  pace  while  the 
testees  simply  listen.  The  second  time  it  is  read  at  conversational  pace,  but  chunked 
into  natural  sections  of  about  5-6  words  which  are  not  repeated.  This  kind  of 
task  meets  the  two  naturalness  criteria  for  natural  language  processing  tasks,  viz. 
it  requires  the  processing  of  temporally  constrained  sequences  of  linguistic  mate- 
rial  and,  in  order  to  divide  the  stream  of  speech  into  identifiable  chunks  for 
writing,  it  requires  an  understanding  of  the  meaning  of  what  was  heard  (Oller, 
1979:39). 

The  dictation  passages  were  taken  from  current  Grade  7 textbooks.  A set  of 
criteria  was  drawn  up  jointly  by  the  English  and  N Sotho  team  members  for  the 
marking  of  the  dictation  passages.  Spelling  and  punctuation  were  also  taken  into 
account.  For  the  N Sotho  dictation,  words  that  were  written  conjunctively  instead 
of  disjunctively  were  accepted  as  correct,  provided  they  were  spelled  correctly. 

Reading  comprehension 

N Sotho  and  English  reading  proficiency  was  operationally  defined  as  proficien- 
cy  obtained  in  a reading  comprehension  test  where  a combination  of  test  items 
was  used  for  both  narrative  and  expository  texts.  The  texts  were  taken  from  exist- 
ing  Grade  7 textbooks.  The  test  items  that  were  designed  included  multiple  choice 
questions  of  an  inferential  nature,  vocabulary  questions,  cloze  items,  identifying 
referents  of  anaphoric  items,  and  questions  involving  graphic  information,  e.g. 
maps  and  graphs. 

Reading  rate 

During  the  reading  test  an  informal  measure  of  the  learners'  reading  rate  was 
taken.  After  the  test  preliminaries,  the  learners  were  instructed  to  start  reading. 
After  a minute,  they  were  stopped  and  asked  to  circle  the  word  they  had  been 
reading.  Readers  then  continued  the  passage  and  answered  the  questions  that 
followed.  The  number  of  words  read  gave  a rough  indication  of  reading  rate. 
Because  it  is  difficult  to  accurately  assess  reading  rate  in  large  groups,  the  scores 
are  treated  with  caution. 
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Administrative  procedures 

The  English  tests  were  administered  first,  in  October.  To  reduce  memory  effects, 
the  N Sotho  tests  were  written  about  3 weeks  after  the  English  tests.  Both  sets  of 
tests  were  written  during  two  periods  allocated  during  school  hours  and  admin- 
istered  by  the  project  researchers.  No  specific  time  limits  were  set  for  completion 
ofthetests. 


5.  Results 

The  data  were  captured  and  analysed  using  SPSS.  Using  the  Cronbach  alpha 
model,  the  reliability  score  for  the  English  tests  was  .74  respectively,  while  the 
alpha  score  for  the  N Sotho  pre-  and  posttests  was  .75.  Given  the  small  scale 
nature  of  the  study,  these  alpha  scores  are  regarded  as  acceptable  (e.g.  George  & 
Mallory,  2003:231). 

Descriptive  statistics  were  used  to  explore  the  first  question,  viz. 

1.  What  are  the  language  and  reading  profiles  of  Grade  7 leamers  from  three 
different  township  schools? 

Table  1 reflects  the  mean,  minimum  and  maximum  scores.  The  quartiles  for  read- 
ing  in  English  show  how  the  distribution  of  learners'  scores  compare  across  the 
schools.  Three  interestingpatterns  emerged.  Firstly,  as  can  be  seen  from  the  table, 
the  two  township  schools  performed  similarly  in  reading,  in  both  N Sotho  and 
English,  with  both  schools  showing  low  reading  levels.  In  contrast,  the  private 
school  far  outperformed  the  intervention  schools  on  language  and  reading  scores 
and  the  learners  also  had  faster  reading  rates.  Secondly,  one  notes  the  large  dis- 
crepancy  between  performance  on  the  language  and  reading  tests  in  the  town- 
ship  schools  (almost  30%  in  the  case  of  N Sotho).  Thirdly,  the  N Sotho  reading 
scores  in  the  township  schools  lagged  behind  the  English  reading  scores,  and  the 
learners  read  more  slowly  in  N Sotho  than  in  English. 

Table  2 shows  the  breakdown  of  performance  in  the  different  components  of  the 
reading  test.  There  are  two  trends  to  note  here.  Firstly,  there  is  again  the  striking 
differential  performance  on  all  components  of  the  reading  test  between  the  town- 
ship  schools  on  the  one  hand  and  the  private  school  on  the  other  hand.  The  town- 
ship  schools  show  low  and,  in  general,  fairly  similar  scores  on  all  the  measures;  in 
contrast,  the  private  school  scored  well  on  all  the  measures.  The  inferendng,  cloze 
and  anaphoric  components  of  the  tests  are  all  aspects  of  reading  that  require  read- 
ers  to  perceive  connections  between  text  units  in  order  to  make  sense  of  the  text  as 
a whole,  yet  it  is  these  aspects  that  posed  challenges  for  the  learners  in  the  town- 
ship  schools.  Learners  who  struggle  with  these  kinds  of  test  items  are  usually 
leamers  who  have  not  yet  learned  to  be  strategic,  meaning-making  readers. 
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Table  1 : Grade  7 comparison  of  mean  percentages  across  the  schools 

November  2006 

Township  School  P 

Township  School  B 

Private  School 

(n  = 54) 

(n  = 50) 

(n  = 25) 

Average  age 

13.6 

13.6 

13.4 

(Range  of  years) 

(11-16) 

(11-16) 

(12-14) 

Mean  NS  dictation  % 

64.5 

67.7 

SD 

27.04 

25.65 

— 

Minimum 

0 

0 

Maximum 

97 

98 

Mean  NS  reading  comprehension  % 

36.06 

38.1 

SD 

21.57 

19.62 

— 

Minimum 

2 

6 

Maximum 

91 

82 

Mean  English  dictation  % 

63.4 

55.2 

92.4 

SD 

30.55 

33.54 

13.14 

Minimum 

0 

2 

45.2 

Maximum 

98 

100 

100 

Mean  English  reading  comprehension  % 

46.2 

44.5 

80.4 

SD 

19.89 

19.99 

11.92 

Minimum 

11 

9 

52.3 

Maximum 

97 

85 

95.2 

Percentiles  25 

30.9 

30.9 

72.6 

50 

42.8 

39.2 

80.9 

75 

57.1 

62.5 

90.4 

Mean  Reading  rate  N Sotho 

102 

106 

— 

(words  per  minute)  English 

132 

131 

169 

Secondly,  one  notes  in  the  township  schools  similarities  in  performance  be- 
tween  scores  in  N Sotho  and  English  on  component  parts  of  the  test.  For  example, 
learners'  inferencing  abilities  were  similar,  irrespective  of  whether  they  were 
answering  an  inference  question  in  N Sotho  or  in  English.  The  cloze  and  ana- 
phoric  components,  items  that  require  fairly  close  attention  to  textual  details  for 
meaning  construction  within  and  across  sentence  boundaries,  yielded  relatively 
better  performance  in  English  than  in  N Sotho.  The  only  component  of  the  read- 
ing  test  that  township  leamers  coped  with  relatively  adequately  was  the  section 
relating  to  questions  about  a graph  (58%  and  60%  respectively).  This  was  also  the 
easiest  component  of  the  reading  test  for  the  learners  at  the  private  school,  and 
they  sailed  through  it  with  a 93%  average. 

To  further  explore  the  relation  between  performance  in  reading  in  the  two 
languages  in  the  township  schools,  a Pearson  Product  Moment  correlation  was 
applied,  yielding  a robust  and  highly  significant  correlation  between  reading  in 
N Sotho  and  reading  in  English:  r = .17  (p  < 0.0001).  In  other  words,  if  learners 
were  good  at  reading  in  one  language,  they  tended  to  be  good  at  reading  in  the 
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Table  2:  Grade  7 Comparison  of  components  of  English  reading 
comprehension  across  the  schools  November  2006 


Township  School  P 

Township  School  B 

Private  School 

NS 

Eng 

NS 

Eng 

Mean  reading  comprehension  % 

36.06 

46.2 

38.1 

44.5 

80.4 

Components  of  reading  test  % 

Vocabulary 

47.3 

48.8 

46.3 

42.6 

82.6 

Inferencing 

48.8 

49.7 

50.3 

49.6 

74.6 

Cloze 

27.5 

43.2 

33.6 

41.4 

77.4 

Reading  graphs 

58.9 

58.2 

60.8 

47.3 

93.6 

Anaphoric  resolution 

25.0 

36.6 

18.1 

41.0 

80.5 

other  language;  similarly,  poor  readers  in  one  language  were  aiso  poor  readers  in 
the  other  language. 

The  second  research  question  seeks  to  examine  the  link  between  reading  and 
school-based  performance: 

2.  How  does  reading  performance  in  Grade  7 relate  to  academic  perform- 
ance? 

In  order  to  address  this  queshon,  the  leamers'  scores  in  the  Grade  7 end-of-year 
examinations  were  obtained  and  a mean  score  computed  from  the  eight  subjects 
(English,  N Sotho,  Afrikaans,  Maths,  Natural  Science,  Social  Science,  Life  Orien- 
tation  and  Technology).  The  learners  were  then  placed  into  four  achievement 
categories  used  by  the  Department  of  Education,  viz.  NotAchieved  (0-39%),  Partial- 
lyAchieved  (40-49 % ),  Achieved  (50-69%)  and  Outstanding  (70-100%).  The  means  for 
L1  and  L2  language  and  reading  of  the  learners  in  each  of  these  four  levels  were 
then  tabulated.  The  results  are  shown  in  Table  3.  It  is  clear  from  the  table,  firstly, 
that  across  the  schools  there  is  a trend  of  increased  language  and  reading  ability 
associated  with  academic  category.  Learners  in  the  Not  Achieved  category  had 
much  lower  reading  scores  than  learners  in  the  Partially  Achieved  category,  who 
in  tum  had  lower  reading  scores  than  those  in  the  Achieved  group;  learners  in  the 
Outstajiding  group  were  all  competent  readers. 

Secondly,  the  trend  of  increased  language  and  reading  ability  associated  with 
academic  category  operated  at  two  different  levels  according  to  the  type  of  school. 
In  other  words,  the  notion  of  'good'  and  'poor ' reader  was  relative  to  the  school 
context.  The  Not  Achieved  learners  at  the  private  school  were  reading  at  higher 
levels  than  the  learners  in  the  Achieved  category  in  the  township  schools.  This 
differential  trend  is  clearly  seen  in  Figure  1,  where  the  mean  reading  scores  are 
given  of  learners  in  each  academic  category  across  the  three  schools.  Table  3 also 
clearly  illustrates  two  other  features  previously  identified,  namely,  the  lower  N 
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Table  3:  2006  Comparison  of  L1  and  L2  language  and  reading  proficiency 

across  the  four  academic  achievement  categories 


Academic 

Language  & reading 

Township  School  B 

Township  School  P 

Private  school 

category 

assessment 

Mean 

Mean 

Mean 

Not  Achieved 

L1  language 

24.1 

37.6 

- 

L1  reading 

13.3 

12.2 

- 

L2  language 

11.9 

17.6 

79.7 

L2  reading 

19.04 

20.2 

63.6 

Partly  Achieved 

L1  language 

48.4 

58.6 

- 

L1  reading 

26.3 

30.3 

- 

L2  language 

27.1 

50.6 

84.8 

L2  reading 

32.9 

38.6 

70.8 

Achieved 

L1  language 

77.4 

69.1 

- 

L1  reading 

42.3 

39.2 

- 

L2  language 

67.8 

74.1 

96.9 

L2  reading 

48.3 

50.4 

86 

Outstanding 

L1  language 

94.6 

84.3 

- 

L1  reading 

65 

60 

- 

L2  language 

96.7 

87.6 

99.4 

L2  reading 

78.5 

72.3 

91.6 

Sotho  reading  scores  in  relation  to  English  with  each  group,  and  the  wide  gap 
between  the  dictation  and  reading  scores  in  both  languages  in  the  township 
schools. 

It  is  also  instructive  to  see  how  learners  in  the  different  academic  groups 
coped  with  the  various  components  of  the  reading  test,  as  reflected  in  Table  4.  At 
this  point  it  is  important  to  note  that  the  learners  at  the  private  school  finished  the 
reading  test  in  half  the  time  it  took  the  township  learners  to  complete  the  test. 
Several  of  them  made  remarks  afterwards  such  as  "Oh,  mam,  that  was  easy!"  or 
"That  was  fun!  When  are  you  coming  again?"  These  observations  are  supported 
by  the  outcomes  reflected  in  Table  4. 

Here  too  we  see  a trend  of  increased  performance  in  all  the  reading  compo- 
nents  across  the  academic  groups,  with  fairly  large  differences  in  the  township 
schools  in  mean  scores  for  the  inferencing,  cloze  and  anaphoric  components 
between  the  Partly  Achieved  and  the  Achieved  learners,  and  again  between  the 
Achieved  and  the  Outstanding  learners.  There  is  also  a clear  differential  distribu- 
tion  of  mean  scores  across  the  academic  groups  between  the  township  schools 
and  the  private  school.  The  Not  Achieved  and  Partly  Achieved  learners  at  the  private 
school  performed  most  poorly  on  the  cloze  item  (50%),  but  this  was  still  higher 
than  the  Achieved  leamers'  performance  on  the  same  item  in  the  township  schools. 
These  results  clearly  suggest  that  even  learners  who  have  failed  at  the  private 
school  have  achieved  much  higher  levels  of  literacy  accomplishment  than  a great 
many  of  the  learners  who  pass  in  the  township  schools. 
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Figure  1 : Differential  reading  performance  across  the  schools 


Table  4:  Performance  in  components  of  reading  test  across  the 
four  academic  groups 


Not  Achieved 

Partly  Achieved 

Achieved 

Outstanding 

Vocabulary  questions 

School  P 

30.9 

47.4 

52.7 

60 

School  B 

27 

33.3 

45.2 

54.1 

Private 

70.8 

87.5 

86.6 

94.1 

Inferencing  questions 

School  P 

30.9 

35.8 

56.1 

73.3 

School  B 

33.3 

42.1 

51.2 

75 

Private 

62.5 

66.6 

78.8 

83.3 

Cloze  questions 

School  P 

13.4 

33.4 

49.4 

78.8 

School  B 

5.8 

23.3 

46.3 

92.6 

Private 

50 

61.7 

86.6 

94.1 

Anaphoric  resolution 

School  P 

25 

22.1 

40 

67.5 

School  B 

0 

33 

43.3 

69 

Private 

71.8 

75 

83.3 

88 

Reading  a graph 

School  P 

45.7 

6 

60.6 

68 

School  B 

0 

37.2 

52.3 

67 

Private 

80 

90 

97 

100 
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Finally,  to  explore  the  relationship  between  the  language  and  reading  varia- 
bles  (as  independent  or  predictor  variables)  and  academic  performance  (as  the 
dependent  variable),  stepwise  regression  analyses  were  performed  separately  for 
the  township  schools  and  for  the  private  school.  Three  learners  had  marks  that 
were  excessively  influencing  the  models  so  these  leverage  points  were  removed. 
Even  though  the  outcomes  varied  slightly  with  their  removal,  significant  models 
still  obtained  for  both  school  groups.  For  the  township  schools,  using  the  step- 
wise  method,  a significant  model  emerged  (F2m  = 57.0,  p < 0.000).  Adjusted  R 
square  = 0.52.  For  the  private  school,  Reading  Comprehension  was  a significant 
predictor  (F^  ^ = 93.5,  p < 0.000).  Adjusted  R square  = 0.80.  Standardised  Beta 
coefficients  identifying  significant  predictor  variables  are  shown  in  Table  5. 


Table  5:  Predictor  variables  for  exam  performance 


Predictor  variables 

Beta 

Township  schools 

L2  Reading  comprehension 

0.452 

p < 0.0005 

L2  Language 

0.333 

p = 0.001 

Private  school 

L2  Reading  comprehension 

0.900 

p < 0.0005 

In  the  township  schools  English  Reading  Comprehension  alone  accounted  for 
48%  of  the  variance.  The  inclusion  of  English  Language  resulted  in  explaining  an 
additional  5%  of  the  variance.  This  model  accounted  for  52%  of  the  variance.  In 
the  township  schools  N Sotho  Language  and  N Sotho  Reading  Comprehension 
did  not  predict  exam  performance.  In  the  private  school  English  Reading  Com- 
prehension  was  a sufficient  predictor  for  exam  performance,  accounting  for  80% 
of  the  variance.  These  results  confirm  that  for  both  the  township  and  the  private 
schools,  there  is  a strong  relationship  between  reading  ability  and  academic  per- 
formance. 

6.  Discussion 

There  are  three  salient  findings  to  emerge  from  the  data  in  this  study.  Firstly,  there 
is  the  large  discrepancy  in  reading  performance  between  the  township  schools 
and  the  private  school,  with  the  former  showing  disturbingly  low  levels  of  read- 
ing  and  the  latter  showing  consistently  high  levels  of  reading  performance.  It 
must  be  noted  that  Schools  B and  P are  fairly  'typical'  township  schools  so  one 
can  presume  that  their  performance  on  the  language  and  reading  measures  are 
not  uncharacteristic  of  similar  township  schools  - which  constitute  the  bulk  of 
schools  in  South  Africa  (Gustafsson,  2005).  If  after  seven  years  of  primary  school- 
ing  learners  enter  high  schools  with  such  low  literacy  outcomes,  it  is  unlikely 
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that  they  will  be  able  to  catch  up  during  five  years  of  high  school  when  the 
pedagogic  focus  is  on  content  subjects,  not  literacy  development.  The  implica- 
tions  are  clear:  the  success  of  academic  redress  and  equity  lies  in  the  quality  of 
education  in  primary  schools. 

The  second  salient  finding  relates  to  the  consistent  and  robust  relationship 
that  obtained  between  reading  comprehension  and  academic  performance  across 
all  three  schools.  The  better  the  reading  scores,  the  stronger  the  leamers'  academ- 
ic  achievement.  The  implications  are  clear:  in  order  to  improve  academic  per- 
formance  schools  need  to  focus  specifically  on  improvingleamers'  readingskills. 
Primary  schools  have  a major  role  to  play  in  this  regard. 

The  third  salient  finding  is  applicable  to  the  township  schools  and  relates  to 
reading  in  N Sotho.  Reading  in  the  L1  lagged  behind  reading  in  Hnglish,  and 
large  differences  generally  obtained  between  performance  on  the  language  and 
the  reading  tests,  particularly  in  N Sotho.  Although  reading  levels  were  general- 
ly  low  in  both  languages,  the  finding  that  the  leamers  read  somewhat  faster  and 
better  in  English  relative  to  N Sotho  is  perhaps  not  surprising,  considering  that 
most  of  the  reading  activities  that  occur  within  classrooms  tend  to  be  done  in 
English,  the  LoLT.  This  finding  supports  the  view  that,  though  dependent  on 
language,  reading  is  a distinct  ability  which  develops  in  specific  ways.  If  chil- 
dren  are  not  regularly  exposed  to  texts,  they  do  not  become  good  readers.  Read- 
ing  in  their  own  language  did  not  confer  an  advantage  on  the  leamers,  even 
when  they  did  well  on  the  N Sotho  dictation  test.  Having  N Sotho  as  a home 
language  is  not  a sufficient  condition  for  becoming  a reader  in  N Sotho.  The 
implications  are  clear:  decisions  about  language  policy  in  schools  must  take  into 
account  the  fact  that  language  profidency  (in  the  L1  or  an  AL)  is  an  important  but 
not  a sufficient  condition  for  academic  performance. 

In  the  next  section  we  briefly  consider  what  these  findings  suggest  about  the 
nature  of  academic  literacy  development  in  primary  schools,  and  the  dynamics 
underlying  academic  (un)preparedness. 

Implications  for  the  development  ofacademic  literacy  in  schools 
In  all  our  schools  there  is  a general  concem  to  develop  the  language  profidency  of 
learners,  the  assumption  being  that  language  profidency  is  the  gateway  to  success 
at  school,  especially  proficiency  in  the  LoLT.  However,  as  many  studies  have 
found  (e.g.  Cummins,  1991, 2000;  Snow  & Dickinson,  1991;  Snow,  Bums  & Griffin, 
1998),  general  language  profidency  is  not  a consistent  predictor  of  academic  per- 
formance.  Although  many  of  the  learners  in  this  study  performed  quite  well  on 
the  dictation  test  which  taps  into  a more  general  knowledge  of  language,  it  was 
their  performance  on  reading  especially  that  predicted  academic  performance. 

It  is  largely  through  regular  exposure  to  the  'book  language'  of  print  material 
over  many  years  that  leamers  become  rapid  and  fluent  readers  who  use  cues  in 


120 


Ensovoort:  jaargang  1 1,  nommbr  2,  2007 


the  texts  to  construct  meaning.  Through  regular  reading  leamers  are  also  exposed 
to  the  vocabulary,  text  structures  and  conventions  that  characterise  academic 
discourse,  and  thus  they  develop  academic  language  proficiency  that  underpins 
success  in  the  learning  context. 

Three  of  the  basic  building  blocks  for  the  development  of  academic  literacy 
in  primary  schools  are  easy  access  to  a variety  of  books,  opportunity  to  read,  and 
motivation  to  read.  These  factors  tend  to  be  absent  in  the  majority  of  township 
schools  that  serve  poor  communities.  In  our  South  African  context,  many  learn- 
ers  don't  read  well  simply  because  they  don't  read  enough:  the  schools  they 
attend  do  not  make  reading  a real  priority,  very  little  time  is  spent  on  reading, 
and  print  resources  with  which  to  inculcate  good  reading  practices  are  virtual- 
ly  absent.  Attention  to  reading  and  nurturing  a positive  attitude  towards  read- 
ing  is  vitally  important  for  developing  automaticity  on  which  the  development 
of  more  sophisticated  comprehension  processing  can  build.  The  slow  reading 
rate  of  the  learners  in  the  township  schools  is  indicative  of  learners  who  do 
very  little  reading  of  extended  texts.  Only  regular,  extensive  reading  leads  to 
increased  reading  speed,  which  leads  to  improved  working  memory  capacity, 
which  in  turn  facilitates  comprehension  processing  (Walczyk,  Marsiglia,  Johns 
& Bryan,  2004). 

Based  on  his  research  into  the  development  of  comprehension  and  represen- 
tation,  Van  den  Broek  (1997:321)  states  that  "(o)ne  of  the  most  essential  aspects  of 
our  understanding  of  the  world  ...  is  the  ability  to  recognise  the  relations  be- 
tween  the  events  that  we  encounter  ".  The  same  prindple  applies  to  reading  and 
is  consistently  reflected  in  the  reading  results  from  this  study : the  learners  at  all 
three  schools  who  demonstrated  an  ability  to  infer  the  relations  between  the 
events  they  encountered  in  their  texts,  as  reflected  especially  in  the  inference, 
cloze  and  anaphoric  items,  were  learners  who  understood  their  text  world.  This 
ability  to  focus  attention  on  relevant  information  and  to  build  connections  is 
fundamental  to  academic  literacy.  This  suggests  an  ability  to  construct  meaning 
at  a deeper  level  of  processingby  locating  details  and  utilising  relevant  textual 
information.  Not  only  did  the  good  readers  understand  their  text  world  better 
than  their  peers  who  performed  poorly  on  these  components,  they  were  also 
performingbetter  academically  because  this  ability  gave  them  the  potential  to 
integrate  their  text  world  into  their  knowledge  bases  and  thereby  acquire  new 
knowledge. 

According  to  Cummins'  Linguistic  Interdependence  Hypothesis  (1991, 2000), 
academic  literacy  operations  and  constructs  transfer  across  languages  and  do  not 
have  to  be  "relearnt"  in  another  language.  It  is  usually  assumed  that  the  direction 
of  influence  is  from  the  home  language  to  an  AL,  and  indeed  most  the  findings 
from  the  developed  world  support  this  assumption  (cf.  Cummins,  2000).  The 
findings  from  the  township  schools  lend  some  support  for  this  hypothesis,  as 
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seen  in  the  strong  correlation  that  obtained  between  reading  in  N Sotho  and 
English  in  the  township  schools.  However,  the  fact  that  the  stronger  readers  were 
consistently  stronger  in  reading  in  English  than  they  were  in  N Sotho  suggests 
that  the  direction  of  influence  is  coming  from  English,  not  the  home  language. 
The  findings  from  this  study  suggest  that  for  many  learners  academic  literacy  is 
more  strongly  developed  in  English  than  in  the  home  language. 

Based  on  her  research  into  the  teaching  of  Southem  Sotho  as  home  language 
at  primary  school,  Smyth  (2002:93, 194-195)  argues  that  learners  should  be  given 
the  opportunity  to  develop  academic  literacy  in  their  home  languages  in  order  to 
provide  a sound  conceptual  and  linguistic  basis  for  future  learning  across  all 
content  leaming  areas.  This  is  not  happening  yet,  despite  school  language  policy 
that,  theoretically,  makes  this  possible.  The  lower  reading  levels  and  slower  read- 
ing  rates  in  N Sotho  in  the  current  study  are  strongly  indicative  that  not  enough 
reading  is  being  done  in  N Sotho.  One  of  the  factors  that  may  contribute  to  this 
situation  lies  in  the  diglossic  differences  between  spoken  and  written  N Sotho. 
The  N Sotho  that  is  spoken  in  the  Pretoria  area  (Sesotho  sa  Pretoria)  is  different  in 
many  respects  from  the  language  of  standard  N Sotho  (Sesotho  sa  Leboa).  Leam- 
ers  do  not  have  enough  exposure  to  written  N Sotho  and  opportunities  to  devel- 
op  proficiency  in  interpreting  and  using  written  forms  of  the  language.  The  lack 
of  resources  exacerbates  this  situation.  For  example,  since  the  start  of  the  interven- 
tion  project  the  iibrary  at  School  B has  grown  from  200  books  to  over  3,500  books, 
yet  there  are  only  139  titles  of  N Sotho  books  in  the  library,  despite  efforts  to 
purchase  more  N Sotho  books.  Most  of  these  are  storybooks  intended  for  children 
under  the  age  of  about  10  years:  teenage  fiction  and  non-fiction  in  N Sotho  is 
practically  non-existent. 

The  findings  from  the  private  school  indicate  that  when  conditions  in  a 
school  are  conducive  to  leaming,  very  high  literacy  levels  can  be  achieved,  even 
when  the  LoLT  is  not  the  L1  of  the  learners.  This  suggests  that  in  multilingual 
learning  contexts  in  which  print  materials  in  the  L1  are  scarce,  quality  of  school- 
ing  is  a stronger  determinant  of  the  development  of  academic  language  profi- 
ciency  than  the  language  in  which  and  through  which  such  development  oc- 
curs. 

Shedding light  on  thecauses  ofacademic  'unpreparedness’ 

Given  the  strong  relationship  that  prevails  worldwide  between  socioeconomic 
factors  and  school  achievement  (e.g.  Allington,  2002;  Bradley  & Corwyn,  2002),  it 
could  be  argued  that  the  discrepancies  in  performance  between  the  two  groups 
of  schools  in  this  study  are  not  surprising,  given  their  socioeconomic  differences. 
It  is  not  easy  educating  poor  children,  not  because  children  from  poor  homes  are 
inherently  weaker  than  children  from  middle  class  homes,  but  because  poor 
children  attend  schools  that  tend  to  be  poorly  resourced  and  managed,  with 
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large  classes  and  fewer  well  qualified  teachers.  Poor  children  also  come  from 
homes  that  contain  few  literacy  resources  and  whose  parents  have  lower  literacy 
levels.  Combined,  these  SES-driven  home  and  school  factors  conspire  to  create 
barriers  to  leaming  and  literacy  accomplishments. 

Rather  than  see  SES  as  a moderator  variable  that  influences  a dependent 
variable  such  as  school  outcomes,  one  can  consider  instead  what  factors  might 
mediate  the  effects  of  poverty.  Unintentionally,  poor  schools  may  be  complicit 
in  their  learners'  poor  performance.  Before  the  start  of  the  intervention  project, 
there  was  very  little  reading  happening  inside  or  outside  the  classrooms  in  the 
two  township  schools  in  this  study,  and  very  few  books  to  which  the  learners 
had  access.  The  classrooms  were  characterised  by  an  absence  of  print-based 
material,  reading  homework  was  non-existent  and  learners'  reading  develop- 
ment  was  not  monitored.  Reading  instruction  in  the  early  grades  basically  con- 
sisted  of  learners  reading  lists  of  syllables  and  words  off  the  blackboard.  Learn- 
ers  had  no  exposure  to  storybooks  to  practise  their  reading  skills  in  N Sotho  or 
English,  or  to  discover  the  joy  of  reading.  This  was  in  strong  contrast  to  the 
private  school.  Admittedly  the  latter  school  is  more  affluent,  its  classrooms  are 
well  resourced  and  it  has  much  smaller  class  sizes.  However,  in  addition,  the 
school  is  well  run,  classroom  time  is  well  managed,  reading  is  a priority,  much 
'time  on  task'  is  devoted  to  readingand  writing  activities,  high  standards  are 
expected,  learners  are  regularly  assessed  and  learners  with  reading  problems 
are  identified  and  given  attention. 

Approaches  that  seek  to  identify  factors  that  mediate  the  effects  of  disad  van- 
tage  have  given  rise  to  studies  that  identify  'resilient'  schools  - schools  who  de- 
spite  their  disadvantaged  circumstances  manage  to  achieve  high  academic  stand- 
ards  (e.g.  Wharton-McDonald  et  al.,  1998;  Taylor  et  al.,  2000).  Since  schools  cannot 
change  the  socioeconomic  status  of  the  communities  they  serve,  they  should 
change  themselves  by  becoming  strong  sites  of  literacy  development.  Even  though 
they  remain  poor  schools,  the  two  township  schools  in  this  study  are  becoming 
'print  rich'  poor  schools  with  a stronger  reading  focus.  Book  resources  are  in- 
creasing,  teachers  are  introducing  more  reading  activities  into  their  classrooms, 
there  is  an  explicit  public  discourse  about  reading  at  the  school,  a 30  minute 
literacy  period  for  all  grades  has  been  built  into  the  daily  timetable,  and  parents 
are  encouraged  to  support  their  children's  literacy  activities  at  home.  In  effect, 
these  schools  are  adopting  some  of  the  characteristics  of  effective  schools  and  in 
so  doing,  they  are  becoming  resilient  schools. 

Even  though  reading  levels  remain  low,  both  schools  have  shown  modest 
improvements  in  reading.  There  are  now  fewer  'nonreaders'  (those  who  score 
below  25%  on  reading  tests)  than  previously.  These  changes  notwithstanding, 
the  low  reading  levels  and  the  reading  backlog  that  has  built  up  over  the  years 
mean  that  the  reading  problems  at  these  schools  will  not  disappear  overnight. 
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Although  reading  accounted  for  48%  of  the  variance  in  academic  performance 
despite  the  poverty,  there  are  still  other  factors  at  these  schools  that  are  influenc- 
ing  academic  performance. 

The  fact  that  it  takes  longer  to  develop  academic  language  proficiency  (4-7 
years)  than  general  language  proficiency  must  be  considered  in  conjunction 
with  a phenomenon  well  documented  in  reading  research,  namely  Matthew 
effects  (i.e.  the  rich  get  richer  and  the  poor  get  poorer  situation),  where  good 
readers  getbetter  while  weak  readers  get  weaker  in  relation  to  their  good  reading 
peers  (e.g.  Spear-Swerling  & Stemberg  1996).  Stanovich  (1986)  argues  that  the  gap 
between  good  and  weak  readers  widens  as  learners  move  up  the  educational 
ladder.  The  findings  in  this  study  showed  that  the  reading  gap  between  the  Not 
Achieved  and  the  Outstanding  leamers  was  already  quite  considerable  in  Grade  7. 
Despite  seven  years  of  schooling,  many  of  the  learners  at  the  township  schools 
had  a backlog  of  reading  skills  to  catch  up  at  a time  when  they  were  going  to 
encounter  new  cognitive,  linguistic  and  textual  demands  of  high  school.  It  is 
unlikely  that  disadvantaged  high  schools  will  be  able  to  improve  the  literacy 
levels  of  weak  learners  coming  into  these  schools.  The  importance  of  developing 
sound  reading  abilities  in  primary  schools  cannot  be  overemphasised;  this  will 
minimise  Matthew  effects  and  enable  academic  literacy  to  start  developing  dur- 
ing  the  seven  years  of  primary  schooling. 


7.  Conclusion 

It  is  clear  that  reading  is  a powerful  learning  tool  for  constructing  meaning  and 
acquiring  new  knowledge  in  the  learning  context.  It  also  affords  readers  inde- 
pendent  access  to  information  in  an  increasingly  information-driven  society.  If 
leamers  do  not  start  properly  mastering  this  tool  during  the  primary  school  years, 
then  their  potential  for  success  in  the  learning  context  is  handicapped  from  the 
start.  If  we  use  reading  as  a means  of  "looking  into  the  seeds  of  time"  in  the 
learning  context,  we  can  make  fairly  reliable  predictions  about  which  seeds  are 
likely  to  grow  and  which  will  not:  learners  with  good  reading  ability  will  suc- 
ceed.  Unless  urgent  measures  are  put  into  place  to  develop  sound  reading  abili- 
ties  in  our  primary  schools,  current  concems  about  prospective  students'  ability 
to  cope  in  the  HE  sector  will  continue  to  occupy  us  for  decades. 


124 


Ensovoort:  jaargang  1 1 , nommer  2,  2007 


Acknowledgements 

The  'Readingis  FUNdamental'  project,  from  which  this  research  derives,  is  funded  by  the  DG 
Murray  Trust  and  is  also  supported  by  the  National  Research  Foundation.  Sincere  thanks  are 
due  to  all  the  leamers  and  staff  at  the  three  schools  for  participating  so  generously  and 
willingly  in  the  project.  Thanks  are  also  due  to  Chris  Gilfillan  for  statistical  assistance,  and  to 
the  ALRU  project  team  for  their  steadfast  commitment  and  support:  Sally  Currin,  Nicoline 
Wessels,  Debbie  Mampuru,  Matseleng  Mokhwesana,  Riah  Mabule  and  Kgalabi  Maseko. 


Bibliography 

Allington,  R.L.  2002.  Big  Brotherand  the  National  Reading  Curriculum.  Portsmouth  NH:  Heine- 
mann. 

Bachman,  L.F.  & Palmer,  A.S.  1996.  Language  testing  in  practice.  Hong  Kong:  Oxford  University 
Press. 

Bradley,  R.H.  & Corwyn,  R.F.  2002.  Socioeconomic  status  and  child  development.  Annual 
Review  ofPsychology  53: 371-399. 

Corson,  D.  1997.  The  leaming  and  use  of  academic  English  words.  Language  Learning  47(4):671- 
718. 

Cummins,  J.  1979.  Linguistic  interdependence  and  the  educational  developmentof  bilingual 
children.  Review  ofEducational  Research  49: 222-251 . 

Cummins,  J.  1991 . Conversational  and  academic  language  proficiency  in  bilingual  contexts. 
AILA  Review  8: 75-89. 

Cummins,  J.  2000.  Language,  power  and pedagogy.  Clevedon:  Multilingual  Matters. 

Department  of  Education.  2003.  National  report  on  systemic  evaluation:  Mainstream  Founda- 
tion  Phase.  Pretoria:  Department  of  Education. 

Department  of  Education.  2005.  Systemic  Evaluation  Report:  Intermediate  Phase  Grade  6. 
Pretoria:  Department  of  Education. 

Gee,  J.  1996.  Social  linguistics  and  literacies:  Ideology  in  discourses.  New  York:  Falmer. 

George,  D.  & Mallory,  E 2003.  SPSSfor  Windows.  Boston:  Allyn  & Bacon. 

Griesel,  H.  (ed.).  2006.  Access  and entry  level  benchmarks.  The  National  Benchmark  Tests  Project. 
Pretoria:  Higher  Education  South  Africa. 

Gustafsson,  M.  2005.  The  relationship  between  schooling  inputs  and  outputs  in  South  Africa: 
Methodologies  and  policy  recommendations  based  on  the  2000  SACMEQ  dataseL  [On- 
line].  Available  at  http://www.sacmeq.org/links.htm 

Kucer,  S.B.  2005.  Dimensions  ofliteracy.  Mahwah,  NJ : Lawrence  Erlbaum. 

Mayer,  R.E.  1992.  Guiding  students'  cognitive  processing  of  sdentific  information  in  text.  In:. 
Pressley,  M.,  Harris,  K.R..&  Guthrie,  J.T.  (eds).  1992.  Promoting  academic  competence  and  literacy 
in  school.  San  Diego:  Academic  Press:  243-258. 

Oller,  J.W  1979.  Language  tests  at  school.  London:  Longman. 

Smyth,  A.  2002.  Testing  the  foundations:  An  exploration  of  cognitive  academic  language 

development  in  an  African  home-language  course.  Unpublished  doctoral  thesis.  Johannes- 
burg:  University  of  the  Witwatersrand. 

Snow,  C.E.,  & Dickinson,  D.K.  1991.  Skills  that  aren't  basic  in  a new  conception  of  literacy.  In: 
Jennings,  E.M.  & Purves,  A.C.  (eds.).  Literate  systems  and  individual  lives.  New  York:  State 
University  of  New  York  Press. 

Snow,  C.E.,  Burns,  M.S.  & Griffin,  P 1998.  Preventing  reading  difficulties  in  young  children, 
Washington,  DC:  National  Academy  Press. 

Spear-Sterling  L.  & Stemberg,  R.J.  1996.  Offtrack:  When  poor  readers  become  ‘leaming  disabled'. 
Boulder,  Co:  Westview  Press. 

Stanovich,  K.E.  1986.  Matthew  effects  in  reading:  Some  consequences  of  individual  differences 
in  the  acquisition  of  literacy.  Reading  Research  Quarterly  21 :360-406. 

Taylor,  B.M.,  Pearson,  PD.,  Clark,  K.  & Walpole,  S.  2000.  Effective  schools  and  accomplished 
teachers:  Lessons  about  primary-grade  reading  instruction  in  low-income  schools.  The 
Elementary  School  joumal,  101:121-165. 


Ensovoort:  jaargang  1 1 , nommer  2,  2007 


125 


Thomas,  WP  & Collier,  V.  1997.  School  effedivenessfor  language  minority  students.  Washington, 
DC:  National  Clearinghouse  for  Bilingual  Education. 

Van  den  Broek,  E 1997.  Discovering  the  cement  of  the  universe:  the  development  of  event 
comprehension  from  childhood  to  adulthood.  In:  Van  den  Broek,  E,  Bauer,  EJ.  & Bourg, 
T (eds.).  Developmental  spans  in  event  comprehension  and  representation.  Mahwah,  NJ:  Lawrens 
Erlbaum  Associates. 

Walczyk,  J.,  Marsiglia,  C.S.,  Johns,  A.K.  & Bryan,  K.S.  2004.  Children's  compensations  for 
poorly  automated  readingskills.  Discourse  Processes  37(1):  25-66. 

Wharton-McDonald,  R.,  Pressley,  M.  & Hampston,  J.M.  1998.  Literacy  instruction  in  nine  first- 
grade  classrooms:  Teacher  characteristic  and  student  achievement.  The  Elementary  School 
Joumal,  99:101-128. 


Testing  academic  literacy  over  time:  Is  the  academic  Iiteracy  of 
first  year  students  deteriorating? 

Frans  van  der  Slik 

Radboud  University  of  Nijmegen  6t  Research  associate,  University  of  Pretoria 
Albert  Weideman 
University  of  Pretoria 


Testing  academic  iiteracy  over  time:  Is  the  academic  iiteracy  of  first  year 
students  deteriorating? 

How  much  empirical  evidence  is  therefor  the frequently  expressed  opinion  that  the 
academic  literacy  levels  offirst  year  students  at  South  African  universities  are 
steadily  deteriorating?  Two  tests  ofacademic  literacy  used  widely  in  South  Africa, 
the  Test  ofacademic  literacy  levels  (TALL)  and  its  Afrikaans  counterpart  (TAG) 
may  hold  at  least  a partial  answer  to  this  question.  We  subject  the  administration, 
over  theyears  2005-2007,  ofone  ofthese  tests,  the  Toets  van  akademiese geletterd- 
heidsvlakke  (TAG)  toan  IRT  analysis,  usinga  One-Parameter  Logistic  Model 
(OPLM)  package.  The  results  show  that,  ifwe  equalise  the  subsequent  tests  in 
terms  ofthefirst  administration,  there  is  evidence  that  is  contrary  to  thepopular 
opinion.  More  importantly,  however,  usingan  OPLM  analysis  enables  us  to  make 
more  responsïble  decisions  derivedfrom  test  results,  and  so  make  our  tests  not 
only  theoretically  more  defensible,  but  also  more  accountable  to  a larger  public. 


1 . Introduction 

The  debate  about  academic  literacy  in  South  Africa  is  situated  at  the  interface 
between  school  and  university  education,  or,  in  official  terms,  general  and  fur- 
ther  education  on  the  one  hand,  and  higher  education  on  the  other.  More  often 
than  not,  questions  are  raised  in  terms  of  the  readiness  of  students  about  to  enter 
institutions  of  higher  education,  and  specifically  about  their  preparedness  in 
terms  of  their  ability  to  understand  and  use  academic  language  within  this  new 
environment.  It  is  therefore  not  surprising  that  there  is  already  substantial  expe- 
rience  in  South  Africa  on  the  design  and  use  of  tests  of  academic  literacy  both  for 
access  and  placement  purposes  (cf.  Cliff,  Yeld  & Hanslo,  2003;  Cliff  & Yeld,  2006; 
Visser  & Hanslo  2005;  Weideman,  2003,  Van  der  Slik  & Weideman  2005, 2007;  Van 
Dyk  & Weideman  2004a,  2004b).  In  the  discussion  that  follows,  the  measurement 
of  academic  literacy  levels  is  understood  to  refer  to  the  assessment  of  the  ability  by 
students  to  use  language  at  the  appropriate  and  desired  level  within  the  academ- 
ic  community,  or  their  level  of  competence  in  academic  discourse  and  its  conven- 
tions,  as  this  is  defined  in  the  work  referred  to  here  (especially  Cliff  & Yeld,  2006 
and  Van  Dyk  & Weideman,  2004a). 
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The  interest  in  academic  literacy  levels  is  not  confined  to  scholarly  attention 
and  investigation.  It  engages  both  experts  and  lay  people  in  equal  measure.  A 
popularly  expressed  opinion  would  have  it,  for  example,  that  the  language  abil- 
ities  of  our  students  are  steadily  decreasing  over  time.  In  South  Africa,  such  atti- 
tudes  are  fed  by  occasional  fairly  sensationalist  press  reports  of  lower  (and  by 
implication  lowering)  literacy  levels  among  pre-university  learners  (cf.  for  exam- 
ple  Rademeyer,  2007).  Without  much  ado,  'low'  scores  are  interpreted  as  decreas- 
ing  ability.  The  question  is  almost  never  asked  whether  the  scores  have  not  per- 
haps  been  as  low  as  this  for  some  time.  Phrased  differently:  a chronic  problem  is 
not  the  same  as  standards  that  are  lowering. 

Furthermore,  it  often  escapes  the  readers  of  these  reports  that  some  of  them 
have  as  their  origin  the  testing  of  academic  and  other  forms  of  literacy  by  those 
producing  commercially  designed  tests.  Readers  are  not  told,  in  other  words, 
that  those  with  whom  the  'evidence'  originates  may  have  a financial  interest  in 
the  results  of  these  kinds  of  report.  Dwindling  language  ability  among  the  younger 
generation  is  an  opinion  akin  to  a number  of  those  that  Widdowson  (2005: 15f.) 
discusses  as  'folklinguistics'.  In  the  perception  of  those  involved,  such  strongly 
held  opinions  find  more  than  adequate  evidence  in  their  everyday  experience.  It 
is  of  course  so  that  theoretical  analysis  ignores  naïve  experience  at  its  peril.  Yet  in 
the  present  case  one  would  do  well  to  ask:  is  it  indeed  a matter  of  something 
experienced  intuitively  as  almost  self-evident,  or  could  these  merely  be  deeply 
held  prejudices  and  biases  that  are  masquerading  as  observations  that  are  backed 
up  by  sufficient  evidence? 

The  current  paper  examines  the  question  of  decreasing  levels  of  academic 
literacy  obliquely,  with  reference  to  a number  of  tests  conducted  over  time  at 
North-West  University  (NW)  and  the  Universities  of  Pretoria  (UP)  and  Stellen- 
bosch  (US).  It  belongs  to  a series  of  investigations  that  we  have  done  to  determine 
the  stability  and  robustness  of  the  tests  both  across  various  administrations  with- 
in  the  participating  institutions  (Weideman  & Van  der  Slik,  2007)  and  over  time. 
The  tests  in  question  are  the  Test  ofacadetnic  literacy  levels  (TALL)  and  its  Afrikaans 
counterpart,  the  Toets  van  akademiese  geletterdheidsvlakke  (TAG).  The  purpose  of 
these  investigations  is  to  ensure  in  the  first  instance  a measure  of  theoretical 
defensibility  by  telling,  as  Shohamy  (2001)  exhorts  us  to  do,  "the  story  of  a test". 
This  is  a first  step  towards  the  eventual  public  accountability  that  a test  must  also 
achieve.  In  another  paper  (Weideman  & Van  der  Slik,  2007;  cf.  too  Van  der  Slik  and 
Weideman  2005),  for  example,  we  have  already  checked  if  the  tests  produce  relia- 
ble  outcomes  when  they  are  administered  to  different  populations  of  newly  ar- 
rived  students.  We  plan  to  extend  that  investigation  by  performing  a number  of 
longitudinal  analyses  that  will  inform  us  about  the  ability  of  the  tests  to  predict 
risk  brought  about  by  lower  than  adequate  levels  of  academic  literacy  when  a 
student  enrols  for  study  in  higher  education. 
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Of  course,  an  argument 
could  potentially  be  made  to 
back  up  a decision  to  vary 
the  difficulty  of  the 
admission  tests  over  the 
years  perhaps  for  reasons  of 
capacity,  or  for  other  reasons 
(see  discussion,  below).  But 
by  seeking  to  buiid  in  a 
guarantee,  one  at  least  has 
control  over  difficulty  levels. 


These  tests  of  academic  literacy  have  now  been  used  at  the  three  different 
universities  mentioned  above  since  2005.  Recently,  the  test  has  also  been  adminis- 
tered  to  new  students  of  the  Medical  Faculty  of  the  University  of  Limpopo.  Since 
the  outcomes  of  the  tests  for  the  years  2006  and  2007  have  now  also  become  avail- 
able,  we  are  currently  in  a position  to  give  more  serious  consideration  to  the 
question  raised  in  the  subtitle  of  this  article. 

Though  this  is  easier  said  than  done,  one  way  of  testing  if  secondary  schooling 
nowadays  tums  out  students  whose  ability  is  growing  worse  as  compared  to 
students  from  previous  years  is  to  compare  their  competence  in  academic  lan- 
guage.  What  is  needed  for  such  a comparison  is  some  Archimedean  point  that  can 
be  used  to  compare  students'  language  abilities,  spedfically  their  academic  litera- 
cy,  over  the  years.  The  tests  of  academic  literacy  levels  referred  to  above  might 
provide  just  such  a fixed  point.  However,  despite  the  fact  that  the  TAG  and  TALL 
have  been  extensively  pretested  on  groups  with  known  academic  language  abili- 
ty,  there  is  no  absolute  guarantee  that  the  difficulty  of  the  tests  has  rernained  con- 
stant  over  the  years.  If,  for  example,  the  difficulty  of  the  tests  has  increased  over  the 
years,  one  might  arrive  at  the  false  conclusion  that  the  academic  literacy  of  first 
year  students  has  deteriorated  (while  perhaps  it  has  actualíy  remained  constant  or 
has  even  risen).  Needless  to  say,  the  outcomes  of  these  anaJyses  can  have  important 
consequences,  both  politically  and  for  the  lives  of  individual  students. 

The  latter  point  deserves  some  further  elaboration.  Until  now,  the  tests  of 
academic  literacy  referred  to  here  have  been  employed  as  low  to  medium  stakes 
tests.  That  is,  based  on  their  outcomes,  low  performing  students  were  compelled 
to  enrol  for  an  intervention  programme  at  UP  and  NW,  while  in  the  case  of  US 
students  there  is  in  certain  faculties  a gradual  phasing  in  of  such  programmes.  In 
all  of  these  cases,  no  major  disaster  occurs  for  the  students  if,  as  a result  of  having 
taken  a more  difficult  test,  their  academic  literacy  is  underestimated  as  compared 
to  the  academic  literacy  of  students  of  previous  years.  Some  of  the  students  may  in 
such  a case  be  compelled  to  undergo  additional  tuition  that  they  perhaps  did  not 
fully  need.  But  the  picture  will  change  dramatically  if  the  test  should  be  used  as 
an  admissions  test.  By  their  nature,  such  tests  are  high  stakes  tests,  since  they 
partially  determine  access  to  university  education,  and  the  expected  lucrative 
future  earning  power  that  follows  on  this.  In  such  a case  it  seems  impera- 
— — tive  that  some  guarantee  needs  to  be  given  that  students  with  a given  level 
of  academic  literacy  will  have  the  same  likelihood  of  passing  the  test,  inde- 
pendent  of  the  year  in  which  they  took  the  test.1 

One  way  of  designing  such  a guarantee  into  the  process  of  test  devel- 
opment  and  administration  is  to  make  use  of  Item  Response  Theory  (IRT) 
models  rather  than  of  classical  test  theory.  A prerequisite  for  making  use  of 
the  advantages  of  IRT  modelling  is  that  tests  partly  overlap,  i.e.  items  in  test 
of  year  (t)  are  to  be  found  in  exactly  the  same  format  in  the  test  in  year  ( t + 1 ) 
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as  well.  Fortunately,  this  was  the  case  for  items  of  the  TAG  tests  in  2005, 2006,  and 
2007,  and  these  three  administrations  of  the  test  will  therefore  provide  the  basis 
for  our  analysis.  We  are  not  yet  in  a position  to  do  the  same  for  the  other  (English) 
version  of  the  test  (TALL),  since  here  overlap  is  still  either  too  small  or  absent. 
Though  some  of  the  discussion  and  analysis  below  will  therefore  refer  to  both 
TAG  and  TALL,  since  they  are  parallel  tests,  we  envisage  doing  similar  analyses 
on  TALL  once  the  degree  of  overlap  is  sufficient  for  such  an  analysis  to  be  made. 
In  such  a case,  we  would  test  whether  the  findings  presented  here  present  a 
similar  pattern. 


2.  Method 

Population  and  context 

In  January  and  February  of  2005, 2006,  and  2007,  the  academic  literacy  of  all  new 
undergraduate  students  of  the  University  of  Pretoria,  the  Potchefstroom  and  Van- 
derbijlpark  campuses  of  North-West  University,  and  the  University  of  Stellen- 
bosch  was  tested  through  the  administration  of  the  Test  ofacademic  literacy  levels 
(TALL/TAG).  At  the  University  of  Pretoria  and  University  of  North-West,  students 
are  allowed  to  sit  for  either  the  English  (TALL)  or  Afrikaans  (TAG)  test,  and  so 
have  the  freedom  of  choosing  whichever  language  they  feel  more  comfortable 
with  in  the  academic  environment.  At  the  University  of  Stellenbosch,  however, 
students  have  to  take  both  tests.  At  this  university,  the  English  test  was  adminis- 
tered  one  day  after  the  Afrikaans  test.  In  total  17,659  students  participated  (but 
they  do  not  necessarily  represent  different  students:  see:  Table  1,  note);  9,449  took 
the  Afrikaans  test,  while  the  remaining  8,210  students  participated  in  the  English 
version.  See  Table  1 for  a detailed  description. 


The  tests:  TALL  and  TAG,  and  their  design 
The  2005  and  2006  versions  of  TALL  and  TAG  each  consists 
of  120  marks,  distributed  over  seven  subtests  or  sections 
(described  in  Van  Dyk  & Weideman,  2004a  and  2004b,  Wei- 
deman,  2006),  six  of  which  are  in  multiple-choice  format: 
Section  1:  Scrambled  text 
Section  2:  Understanding  graphs  and  visual 
information 

Section  3:  Understanding  texts 
Section  4:  Academic  vocabulary 
Section  5:  Text  types 
Section  6:  Text  editing 

Section  7:  Writing  (handwritten;  marked  and 
scored  only  for  certain  borderline  cases) 


Table  1:  Population  offirstyear  students 


TALL 

UP 

US* 

NW 

Total 

2005 

3 310 

1 729 

135 

5 174 

2006 

3 652 

3 710 

143 

7 505 

2007 

3 905 

4 165 

140 

8 210 

TAG 

UP 

US* 

NW 

Total 

2005 

2 701 

1 702 

2 521 

6 924 

2006 

2 547 

3 703 

2 650 

8 900 

2007 

2 582 

4 160 

2 707 

9 449 

* Note:  Stellenbosch  students  took  the  TALL  the  day  after 
they  took  the  TAG. 
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The  2007  versions  of  TALL  and  TAG  each  consists  of  100  marks,  distributed  over 
the  first  six  subtests  or  sections  which  are  in  multiple-choice  format.  Section  7 was 
omitted  from  2007  on;  borderline  cases,  who  are  identified  by  statistical  means, 
are  allowed  to  take  another  test,  the  results  of  which  are  used  to  decide  if  the 
student  has  passed  or  failed  the  test,  and  has  risk  in  terms  of  academic  language. 

Students  have  60  minutes  to  complete  the  test,  and  they  earn  a maximum  of 
100  marks  (some  items  counting2  or  3 instead  of  1).  In  another  paper  (Van  der  Slik 
& Weideman,  2005;  cf.  too  Weideman  & Van  der  Slik,  2007),  the  determination  of 
the  cut-off  point  has  been  discussed  extensively.  We  will  return  to  the  issue  of  cut- 
off  points  below,  where  we  will  evaluate  them  in  light  of  the  outcomes  of  the  IRT- 
based  analyses. 


3.  Analyses 

In  order  to  perform  ERT  analyses,  we  make  use  here  of  the  One-Parameter  Logistic 
Model  (OPLM)  package  developed  by  Norman  Verhelst  and  his  colleagues  at 
CITO  in  the  Netherlands  (Verhelst,  Glas  & Verstralen,  1995).  IRT  analyses  repre- 
sent  an  ability  of  persons  such  as,  for  example,  academic  literacy,  in  a mathemat- 
ical  model.  In  an  IRT  analysis,  the  ability  is  usually  denoted  by  the  Greek  letter 
theta  (0).  Persons  with  a high  0 (ability)  are  expected  to  have  a high  chance  to  give 
correct  responses  to  difficult  items,  while  persons  with  low  ability  are  expected  to 
have  a low  likelihood  to  answer  difficult  items  correctly.  The  attractiveness  of  IRT 
modelling-  as  compared  to,  for  example,  Guttman  scaling-is  that  persons  who 
get  difficult  items  correct,  still  have  the  likelihood  to  respond  incorrectly  to  less 
difficult  items.  Similarly,  less  able  persons  have  a chance  to  respond  correctly  to 
difficult  items.  Guttman  scaling  does  not  allow  for  such  "inconsistencies". 

Various  mathematical  models  may  be  used  to  represent  the  characteristics  just 
described,  but  the  critically  important  consideration  for  choosing  the  appropriate 
model  would,  no  doubt,  be  the  degree  to  which  it  would  fit  the  data.  In  respect  of 
IRT  modelling,  various  fit  measures  can  be  employed  to  evaluate  the  model  against 
the  data,  but  a logistic  curve  has  the  most  attractive  set  of  features.  One  of  the  main 
models  in  IRT  analyses  is  the  Rasch-model.  A iess  attractive  feature  of  this  model, 
however,  is  that  it  assumes  that  all  iterris  measuring  a specific  ability  have  the  same 
discrimination  index.  The  One-Parameter  Model  that  we  are  using  in  the  current 
analysis  relaxes  this  restriction  by  allowing  discrimination  indices  to  vary.  It  may 
thus  represent  the  data  better,  since  it  is  well  known  in  classical  test  theory  that  test 
items  may  vary  rather  considerably  as  regards  their  discriminating  power.  One 
decisive  advantage  of  IRT  analyses  over  classical  test  theory,  however,  is  that  they 
can  cope  with  incomplete  designs.  That  is:  the  program  can  deal  with  different 
persons  responding  to  different  sets  of  items  (or  tests  for  that  matter).  See  Figure  1. 
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Figure  1:  Persons  by  items  matrix 


In  Figure  1,  the  rows  represent  persons  taking  the  various  tests,  and  the  items  are 
represented  in  the  columns.  This  is  a diagrammatic  representation  of  three  tests 
which  overlap  in  part,  i.e.  items  can  be  found  in  different  tests  and  answered  by 
different  persons.  For  example,  test  item  12  is  not  just  in  Test  2005,  but  is  found  in 
Test  2007  as  well,  whereas  items  13  and  14  are  found  in  all  three  tests.  Note  that,  for 
example,  items  1 to  4 can  only  be  found  in  Test  2005.  In  fact,  Figure  1 represents  the 
design  we  are  working  with  in  the  present  study.  Note  also  that,  since  items  may 
occur  more  than  once,  the  number  of  unique  items  is  smaller  than  the  sum  of  the 
test  items.  In  the  present  situation,  155  unique  items  are  involved,  whereas  the 
total  number  of  items  is  62  + 62  + 63  = 187. 

In  case  tests  are  partly  overlapping  (and  therefore  linked),  the  OPLM  program 
is  able  to  estimate  an  ability  distribution  in  which  the  item  parameters  can  be 
estimated  independent  of  the  characteristics  of  the  population.  Rather  than  the 
difficulty  of  the  items,  it  is  the  likelihood  of  correct  answers,  taking  into  account 
a person's  ability,  that  matters.  As  a consequence,  the  ability  distribution  can  be 
used  to  equate  different  tests  in  such  a way  that  cut-off  points  for  the  tests  reflect 
equal  ability,  or,  in  this  case,  levels  of  academic  literacy. 


4.  Results 

Description  ofthe  sample 

Table  2 depicts  the  outcomes  at  scale  level  for  TALL.  As  can  be  seen,  there  is  a 
general  trend  of  decreasing  mean  scores  for  the  three  universities  included  in  this 
study.  Simultaneously,  the  cut-off  points  were  set  to  a lower  level  each  year.  As  a 
result,  the  percentages  of  students  who  failed  to  pass  the  TALL  remained  more  or 


132 


Ensovoort:  jaargang  1 1 , nommer  2,  2007 


Table  2:  Descriptive  statistics  of  TALL 


less  constant  for  UE  In  the  case  of  North-West 
specifically,  but  also  to  some  extent  at  the  Univer- 
sity  of  Pretoria,  one  of  the  deliberate  reasons  of- 
fered  for  a specific  annual  cut-off  point  is  teach- 
ing  capacity  (cf.  Van  der  Slik  and  Weideman,  2005 
and  2007). 

Table  3 provides  the  outcomes  for  TAG.  It  can 
be  seen  that  the  general  trends  observed  for  TALL 
are  also  found  in  TAG.  If  one  looks  only  at  these 
numbers,  the  academic  literacy  of  newly  arrived 
students  at  the  University  of  Pretoria,  the  Uni- 
versity  of  Stellenbosch,  and  of  North-West  ap- 
pears  to  have  deteriorated  over  the  years,  thus 
providing  an  affirmati  ve  answer  to  the  question 
posed  in  the  subtitle  of  this  paper.  These  outcomes 
can  also  be  visualized  as  in  Figures  2 and  3. 

When  Figures  2 and  3 are  considered  in  isola- 
tion,  one  might  be  tempted  to  conclude  that 
something  is  indeed  wrong  with  the  academic 
literacy  of  newly  arrived  students,  since  the  out- 
comes  consistently  demonstrate  a declining  abil- 
ity  for  the  three  universities  involved,  not  just 
for  the  students  taking  the  English  test  (TALL), 
but  also  for  those  who  took  the  Afrikaans  test 
(TAG). 

But  is  this  actually  what  happened?  We  have 
tested  this  hypothesis  by  means  of  OPLM.  As  was 
remarked  above,  we  could,  unfortunately,  per- 
form  these  analyses  only  for  the  Afrikaans  test, 
because  only  these  tests  were  linked,  by  partial 
overlap,  in  the  manner  we  have  described  above. 
We  did  so  by  taking  the  TAG  2005  outcomes  as 
the  reference  or  Archimedean  point.  That  is:  we 
took  mean  ability  associated  with  the  proportion 
that  has  passed  the  test  as  the  starting  point  for 
each  university  separately.  OPLM  has,  in  other 
words,  made  it  possible  for  us  to  show  in  detail 
how  ability  scores  are  associated  with  test  scores  in  2005, 2006  and  2007.  By  trans- 
posing  the  2005  mean  ability  scores  onto  the  2006  and  2007  test  scores,  we  were 
able  to  estimate  the  proportion  of  students  that  would  have  passed  these  2006  and 
2007  tests,  assuming  ability  equal  to  that  of  2005.  In  such  an  analysis,  2005  is  thus 


UP 

US 

NW 

MEAN  (range  1-100) 

2005 

71.75 

76.89 

59.70 

2006 

64.32 

68.46 

56.27 

2007 

61.11 

64.98 

50.44 

Cut-off  point 

2005 

68.5 

68.5 

67.5 

2006 

55.5 

55.5 

49.5 

2007 

50.5 

57.5 

42.5 

Percentage  failed 

2005 

34.26 

22.73 

56.30 

2006 

31.30 

23.02 

34.97 

2007 

31.50 

32.58 

40.00 

Table  3:  Descriptive  statistics  of  TAG 

UP 

US 

NW 

MEAN  (range:  1-100) 

2005 

70.16 

63.15 

63.08 

2006 

60.18 

53.53 

54.07 

2007 

56.66 

51.78 

51.14 

Cut-off  point 

2005 

60.5 

50.5 

55.5 

2006 

50.5 

50.5 

49.5 

2007 

45.5 

42.5 

45.5 

Percentage  failed 

2005 

23.84 

26.85 

31.14 

2006 

25.21 

43.78 

40.35 

2007 

24.98 

33.25 

38.27 
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Figure  2:  Mean  scores  on  TALL  for  2005(1),  2006(2),  and  2007(3) 


accepted  as  the  base  year,  and  the  scores  of  the  subsequent  years  are  interpreted 
with  reference  to  the  mean  scores  of  the  base  year.  In  Figures  4 and  5 we  present 
the  outcomes. 

It  seems  quite  obvious  that  Figure  4 leads  to  a completely  different  conclusion 
than  Figure  3.  Instead  of  a trend  of  declining  ability,  no  apparent  trend  can  be 
observed!  The  cause  of  this  is  that,  instead  of  there  being  a declining  ability  in 
terms  of  academic  literacy  for  students,  the  difficulty  of  the  tests  has  in  fact  in- 
creased.  And  this  increasing  difficulty  over  the  three  years  has  not  been  fully  com- 
pensated  for  by  adjusting  the  cut-off  points.  Figure  5 graphically  represents  this. 
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Figure  4:  Proportions  passed  on  TAG  assuming  equal  difficulty  for  2005(1 ), 
2006(2),  and  2007(3) 


Figure  5:  Cut-off  points  assuming  equal  difficulty  for  2005(1),  2006(2), 
and  2007(3) 


As  can  be  seen  in  Figure  5,  the  cut-off  points  for  the  2005  TAG  test  were  around  48 
marks.  If  we  intend  to  measure  a mean  ability  in  2006  and  2007  that  is  equal  to  the 
one  measured  in  2005,  then  students  would  have  needed  fewer  marks  (between 
42  and  46)  to  make  the  cut-off  point.  Or  put  differently:  If  we  equalise  the  tests  in 
our  analysis  over  the  period  in  question,  by  holding  their  results  steady  in  terms 
of  the  2005  starting  point,  then  we  note  that  the  2005  test  was  in  fact  easier  than  the 
2006  and  2007  tests,  since  a student  who  had  scored  around  48  marks  on  the  2005 
test  would  have  required  a lower  score  (of  between  42  and  46)  on  the  2006  and 
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2007  test  in  order  to  make  the  cut-off  point.  This  implies  that  the  2005  test  was 
easier  than  the  2006  and  2007  TAG  tests. 

The  differences  between  the  cut-off  points  of  the  three  universities  involved 
have  to  do  with  several  factors.  It  can  be  seen,  for  example,  that  the  drop  off 
between  2005  and  2006  for  North-West  students  is  steeper  than  for  Stellenbosch 
students.  This  may  look  odd,  since  they  took  identical  tests;  so  the  drop  off  may  be 
expected  to  be  of  the  same  magnitude.  Two  comments  can  be  made  about  this. 
Firstly,  the  observed  differences  may  in  part  be  coincidental,  resulting  from  meas- 
urement  error.  Secondly,  from  the  point  of  view  of  testing  academic  literacy  lev- 
els,  it  may  be  that  the  students  from  North-West  come  from  a population  that  is 
different  from  the  population  that  the  students  of  Pretoria  and  Stellenbosch  are 
recruited  from,  in  the  sense  that  the  universities  involved  may  be  employing 
different  entry  requirements.  Thus  different  universities  may  land  up  with  dif- 
ferences  among  their  respective  first  year  populations  that  are  greater  than  one 
would  expect  at  first  glance. 


5.  Conclusion 

In  this  article  we  have  tested  if  the  academic  literacy  of  newly  arrived  students  at 
three  universities  has  deteriorated  over  the  period  2005-2007.  A superficial  anal- 
ysis  may  indeed  indicate  that  this  is  the  case.  However,  having  performed  analy- 
ses  by  means  of  the  One-Parameter  Logistic  Model  package  (OPLM),  we  found 
nothing  of  this  kind.  On  the  contrary,  rather  than  a decrease  of  academic  literacy, 
the  analyses  have  shown  that  the  tests  themselves  have  increased  in  difficulty.  It 
is  only  when  we  do  not  fully  compensate  for  this  increased  difficulty  that  it 
appears  as  if  academic  literacy  has  deteriorated.  In  fact,  the  academic  literacy  of 
newly  arrived  Afrikaans  speaking  students  has  proved  to  be  remarkably  constant 
over  the  past  three  years. 

This  study  has  several  limitations  which  have  to  be  addressed  in  future  re- 
search.  First,  the  time  period  under  study  may  be  too  short  to  arrive  at  definite 
answers  regards  the  possibility  of  a decline  in  academic  literacy  in  South  Africa. 
Second,  we  were  only  able  to  analyze  the  results  of  the  Afrikaans  test.  This  is 
unfortunate,  because  Afrikaans  is  taught  mainly  at  formerly  privileged  schools, 
whereas  English  is  taught  at  both  formerly  privileged  and  at  formerly  deprived 
and  extremely  deprived  schools.  Though  the  outcomes  for  the  Afrikaans  and  the 
English  test  of  academic  literacy  levels  at  first  glance  appear  to  be  heading  into  the 
same  direction,  most  of  those  with  experience  in  research  on  language  testing  will 
be  wary  to  rely  solely  on  these  impressions  before  they  can  be  empirically  tested 
and  in  some  way  quantitatively  verified  in  a similar  fashion.  Thirdly,  we  have 
used  only  the  TAG  as  an  indicator  of  academic  literacy  levels.  Other,  similar  tests 
may  and  should  be  used  to  study  trends  in  academic  literacy  levels.  This  is  not  to 
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imply  that  the  results  are  due  to  the  unreliability  of  the  test.  On  the  contrary,  the 
TAG  and  TALL  tests  have  proved  to  be  highly  reliable  over  the  years.  What  we 
mean  is  that  other  measures  than  the  standard  paper-and-pencil  tests  might  also  be 
useful.  Fourth,  an  issue  that  ties  in  with  the  second  point  above:  our  finding  that 
academic  literacy  levels  have  remained  more  or  less  similar  over  the  years  applies 
to  this,  very  specific  and  fairly  select  group  of  students.  As  one  of  the  reviewers  of 
this  article  has  emphasized,  we  have  worked  here  with  a cohort  of  testees  that  is 
circumscribed  in  terms  of  language.  What  if  the  population  is  more  varied  in  terms 
of  first  language,  and  if  the  ability  that  we  have  measured  is  not  as  evenly  distrib- 
uted  across  other  test  populations  as  we  have  assumed  here?  We  would  therefore 
need  to  test  the  kinds  of  conclusions  we  have  reached  here  against  a larger,  and 
perhaps  more  diverse  group.  Finally,  to  take  up  another  point  of  the  same  review- 
er,  the  analysis  challenges  us  as  test  developers  and  as  users  of  the  results  of  these 
kinds  of  tests  to  be  careful  about  assuming  equivalence  among  different  versions  of 
tests.  When  we  interpret  test  results,  we  should  refer  to  test  difficulty,  which  im- 
plies  a measure  of  comparison  that  we  may  not  yet  have  or  may  not  even  have 
planned  for.  We  cannot  simply  interpret  such  results  at  face  value. 

Notwithstanding  these  limitations,  the  outcomes  of  our  analyses  have  made  it 
quite  clear,  we  think,  that  IRT  modelling  is  a useful  tool  to  get  a better  under- 
standing  of  the  difficulty  of  tests  and  test  items.  In  that  sense,  it  may  enable  us  to 
arrive  at  responsible  decisions  that  do  more  justice  to  those  who  take  the  tests,  in 
that  they  are  doubly  accountable,  both  in  being  defensible  in  a theoretical  or 
empirical  sense,  and  in  being  accountable  to  a larger  public  (Weideman,  2006  and 
2007).  This  is  not  only  helpful  for  low  stakes  tests  such  as  TALL  and  TAG,  but  also 
of  critical  importance  when  the  stakes  are  higher,  for  example  in  the  case  of  access 
or  admission  tests.  There  should  really  be  no  dispute  about  the  condition  that  the 
likelihood  of  passing  a test  should  depend  on  students'  ability  as  expressed  in 
terms  of  a score  that  captures  their  level  of  academic  literacy.  IRT  modelling  gives 
us  one  way  of  doing  justice  to  the  expression  of  that  ability  in  the  measurements 
that  are  made  with  similar  instruments  over  time.  The  exceptions  to  this  condi- 
tion  in  our  context  occur  generally  in  the  case  of  tests  that  are  used  for  access  to 
higher  education.  In  such  a case  a political  decision  may  be  taken  where,  say, 
those  belonging  to  a specific,  previously  disadvantaged  group  who  perform  in 
the  top  three  deciles  of  a test  of  academic  literacy  compared  with  their  peers  in 
that  same  group,  may  be  granted  access  to  university  study,  even  though,  in 
comparison  with  others  that  do  not  belong  to  their  group,  they  may  not  have 
made  it.  The  groundbreaking  work  for  making  such  a decision  responsibly  has 
been  done  in  South  Africa  by  the  Alternative  Admissions  Research  Project  of  the 
University  of  Cape  To  wn  (cf.  Cliff  & Yeld,  2006,  Cliff,  Yeld  & Hanslo,  2003,  Visser  & 
Hanslo,  2005).  Though  such  decisions  clearly  need  to  be  taken  on  some  political 
cue,  and  not  one  based  solely  on  a measure  of  ability,  they  would  need  their  own 
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arguments  to  be  defensible  and  justifiable.  But  even  in  this  case,  IRT  modelling 
might  prove  to  be  a valuable  tool  for  underpinning  such  arguments.  It  is  conceiv- 
able,  for  instance,  that  if  there  are  differences  in  academic  literacy  that  are  associ- 
ated  with  membership  of  different  cultural  or  other  groups,  such  differences  will 
not  remain  constant  over  the  years.  IRT  modelling  will  also  be  particularly  suita- 
ble  to  notice  and  monitor  such  changes. 
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Pragmatic  validation  of  a test  of  academic  literacy  at  tertiary  level 

Validity  is  afundamental  consideration  in  language  testing.  Conceptions  of 
validity  have  undergone  a number  ofchanges  over  the  past  decades,  and  validity  is 
now  closely  conneded  with  the  interpretation  oftest  scores.  Validity  remains  an 
abstract  concept,  however,  and  can  only  be  accessed  though  a process  of  validation. 
This  article  illustrates  an  approach  to  the  validation  ofa  test  by  postulating  a 
number  of  claims  regarding  an  administration  ofan  academic  literacy  test  (Toets 
van  Akademiese  Geletterdheidsvlakke)  and  presenting  various  kinds  ofevidence  to 
investigate  these  claims. 


1 . Introduction 

It  is  generally  accepted  that  validity  is  the  central  concept  in  language  assess- 
ment.  The  AERA/APA/NCME  (1999)  test  standards  regard  it  as  the  most  funda- 
mental  consideration  in  developing  and  evaluating  tests.  It  is  a complex  concept, 
which  has  undergone  a number  of  different  interpretations.  At  present  it  is  gen- 
erally  acknowledged  that  validity  is  contextual,  local  and  specific,  pertaining  to 
a specific  use  of  a test,  i.e.  one  asks  whether  the  test  is  valid  for  this  situation.  The 
validity  of  tests  is  determined  though  the  process  of  validation,  a process  of  test 
score  interpretation,  before  the  results  can  be  used  for  a particular  purpose.  In 
order  to  determine  the  validity  of  a test,  a validation  argument  has  to  be  con- 
structed,  on  the  basis  of  which  it  can  be  suggested  whether  the  interpretations 
and  uses  of  the  test  results  are  valid. 

The  purpose  of  this  article  is  to  discuss  current  conceptions  of  validity  and 
then  illustrate  the  process  of  validation  by  constructing  a validation  argument  for 
a widely  used  test  of  academic  literacy.  We  will  propose  a number  of  claims  and 
illustrate  methods  with  which  to  investigate  these,  and  use  the  test  of  academic 
literacy  levels  ( Toets  van  Akademise  Geletterdheidsvlakke)  administered  at  the  Potch- 
efstroom  campus  of  North-West  University  in  2007  to  illustrate  a posteriori  valida- 
tion  procedures. 
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2.  Validity 

The  concept  of  validity  is  not  a fixed  one,  but  has  undergone  different  interpreta- 
tions  over  the  past  50  years.  Two  main  perspectives  can  be  distinguished. 

The  first  is  often  called  the  'traditional'  view,  which  involves  the  question  of 
whether  one  measures  what  one  intends  to  measure.  It  considers  validity  to  be  an 
inherent  attribute  or  characteristic  of  a test,  i.e.  a test  is  valid  if  it  measures  what  it 
claims  tobe  measuring  (Kelley,  1927;  Cattell,  1946;  Lado,  1961).  Three  major  types 
of  validity  are  identified:  criterion-related  validity  (including  concurrent  and 
predictive  validity),  content-related  validity,  and  the  one  introduced  by  Cron- 
bach  and  Meehl  (1955),  construct  validity.  The  traditional  approach  reflects  a 
positivistic  paradigm,  which  assumes  that  a psychologically  real  construct  or 
attribute  exists  in  the  minds  of  the  test  takers  - this  implies  that  if  something  does 
not  exist,  it  cannot  be  measured.  Variations  in  the  attribute  cause  variation  in  test 
scores.  Validity  is  thus  based  on  a causal  theory  of  measurement  (Trout,  1999). 
Reliability,  the  index  of  measurement  consistency,  is  regarded  as  distinct  from 
validity,  and  is  a necessary  condition  for  validity. 

The  second  view  evolved  in  the  1980s,  and  replaced  the  definition  of  three 
validities  with  a single  unified  view  of  validity;  one  which  portrays  construct 
validity  as  central  component  (Chapelle,  1999:  256),  and  regards  content  and 
criterion  validity  as  aspects  of  construct  validity  (Messick,  1989:  20).  Messick's 
(1989)  paper  provided  a seminal  although  somewhat  opaque  exposition  of  this 
view.  This  view  - a more  naturalistic,  interpretative  one  - is  the  most  influential 
current  theory  of  validity.  It  shifted  validity  from  a property  of  a test  to  that  of  test 
score  interpretations.  Validity  is  an  integrated  evaluative  judgementbased  on 
theoretical  and  empirical  evidence,  which  supports  test  score  interpretation  and 
use  (Messick,  1989: 13).  It  is  seen  as  a unitary  but  multifaceted  concept.  Messick 
(1989)  introduced  his  much-quoted  progressive  matrix  - the  types  of  research 
associated  with  validity,  which  involve  the  definition  and  validation  of  the  con- 
struct,  decisions  about  the  individual  (involving  fairness  and  relevance)  [infer- 
ences],  definition  of  social  and  cultural  values,  and  real  world  decisions,  e.g. 
admission  or  placement  [uses].  Messick's  innovation  was  the  introduction  of 
consequential  validity,  i.e.  the  social  consequences  and  effects  of  a test  on  an 
individual  and  society.  Test  consequences  thus  became  a central  part  of  validity. 
Admission  or  placement  decisions  can  have  a major  impact  on  a test  taker,  and 
therefore  aspects  such  as  affect,  washback  and  ethics  are  considered  part  of  conse- 
quential  validity  (cf.  Fulcher  & Davidson,  2007;  Madsen,  1983;  Hughes,  1989; 
Chapelle,  1999).  In  addition,  the  test  context  (including  the  environment,  such  as 
room  temperature  or  seating),  which  may  introduce  construct-irrelevant  vari- 
ance,  can  have  an  impact  on  the  test  scores  (cf.  Fulcher  & Davidson,  2007:  25). 
Brown  (1996: 188)  also  mentions  administration  procedures  and  the  environment 
of  test  administration  as  relevant  factors.  Weir  (2005: 51)  includes  test  taker  charac- 
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teristics  (e.g.  emotional  state,  concentration,  familiarity  with  the  test  task  types)  as 
factors  that  may  influence  test  performance,  and  ultimately  affect  the  validity  of 
the  test  scores. 

This  incorporation  of  a social  dimension  into  validity,  which  can  be  interpret- 
ed  fairly  broadly,  has  not  been  without  controversy,  as  many  critics  argue  that 
validity  does  not  involve  decision-based  interpretations,  but  only  descriptive 
interpretations.  But  the  psychometric  tradition  of  language  testing  has  obscured 
the  role  and  effect  of  language  testing  in  society,  especially  its  sorting  and  gate- 
keeping  roles,  which  ultimately  depend  on  the  policies  and  values  that  underlie 
any  test.  The  practice  of  decision-based  interpretations  has  now  become  part  of 
validity,  although  as  yet  there  is  no  coherent  theory  of  the  social  context  in  cur- 
rent  validity  theory. 

The  current  interpretive  paradigm  thus  allows  a variety  of  data  to  inform  test 
validity.  In  essence,  validity  is  the  truth-value  of  a test.  But  the  question  is:  what 
is  real  or  true?  Ir  uth  remains  a relative  concept,  a question  of  judgement,  a matter 
of  degree,  subject  to  new  or  more  relevant  evidence.  There  is  no  such  thing  as  an 
absolute  answer  to  the  validity  question  (Fulcher  & Davidson,  2007: 18).  This 
view  allows  every  important  test-related  issue  to  be  considered  as  relevant  to  the 
validity  concept  integrated  under  a single  header.  Validity  therefore  involves  a 
chain  of  inferences.  Any  construct  has  to  be  empirically  verifiable,  and  validity 
claims  depend  on  the  evidence  provided  to  support  it.  Fulcher  (1997)  emphasises 
the  fact  that  validity  is  a relative  and  local  affair.  The  argument  for  local  validity  is 
not  current  -it  was  advanced  by  Lado  (1961)  and  Tyler  (1963)  decades  ago.  Weide- 
man  (2006a:  83)  also  stresses  that  the  social  dimension  is  unique  to  each  test  ad- 
ministration.  Tests  are  valid  for  a specific  use,  but  determining  validity  is  an 
ongoing  and  continual  process  (Davies  & Elder,  2005). 

This  second  view  of  validity  gained  prominence  in  language  testing  in  the 
1990s,  when  Bachman  (1990)  introduced  Messick's  ideas  to  language  testingre- 
search.  The  idea  of  validity  as  a unitary  concept  is  now  accepted  by  most  re- 
searchers.  Construct  validity  is  generally  regarded  as  the  overarching  validity 
concept,  but  there  is  still  variation  in  the  use  of  terminology  and  the  sub-types  of 
validity  proposed.  Bachman  and  Palmer  (1996)  introduced  the  overarching  con- 
cept  of  test  usefulness,  but  Bachman  (2005)  later  returned  to  validity  as  metanarra- 
tive.  Weir  (2005: 14)  also  proposed  the  re-introduction  of  validity  as  superordi- 
nate  category,  and  postulated  the  subcategories  of  context  validity,  theory-based 
validity,  scoring  validity  and  external  validity.  McNamara  (2003: 470)  points  out 
that  the  social  dimension  of  validity  is  now  a "prime  topic"  in  language  testing 
debates  (cf.  McNamara  & Roever,  2006). 

In  the  second  view,  reliability  is  no  longer  regarded  as  a separate  quality  of  a 
test, but is part of  overall  validity.  Weir  (2005: 14) says: "...  the  traditional polari- 
sation  of  reliability  and  validity  ...  is  unhelpful  and  reliability  would be better 
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regarded  as  one  form  of  validity  evidence".  Most  researchers  still  regard  reliabil- 
ity  as  important,  as  in  principle  a test  cannot  be  valid  unless  it  is  reliable  (it  can  be 
reliable  but  invalid)  (Alderson,  Clapham  & Wall,  2005: 187). 

The  second  view  of  validity  is,  of  course,  not  without  its  critics.  One  of  the 
reasons  is  that  it  seems  natural  and  instinctive  to  consider  validity  to  be  a feature 
of  a test.  Borsboom,  Mellenburg  and  Van  Heerden  (2004: 3)  say: " . . . we  think  that 
the  argument  that  shifted  the  meaning  of  validity  from  tests  to  score  interpreta- 
tions  is  erroneous".  They  argue  that  there  is  no  reason  why  one  would  conclude 
that  the  term  validity  can  only  be  applied  to  test  score  interpretations.  They  pro- 
pose  a return  to  the  traditional  view  (e.g.  Kelley,  1927: 14),  which  states  that  a test 
is  valid  if  it  measures  what  it  purports  to  measure,  even  though  one  can  only 
validate  interpretations.  They  argue  that  current  accounts  of  validity  only  super- 
ficially  address  theories  of  measurement.  Fulcher  and  Davidson  (2007: 279)  also 
ask:  "Has  this  validity-as-interpretation  mantra  perhaps  become  over-used?  If  a 
test  is  typically  used  for  the  same  inferential  decisions,  over  and  over  again,  and 
if  there  is  no  evidence  that  it  is  being  used  for  the  wrong  decisions,  could  we  not 
speak  to  the  validity  of  that  particular  test  - as  a characteristic  of  it?  Or  must  we  be 
on  constant  guard  for  misuse  of  all  tests?" 

The  view  of  validity  as  interpretation  is  now  widely  accepted.  But  it  is  de- 
pendent  on  test  results  being  used  for  the  purpose  for  which  the  test  is  designed. 
Score  interpretation  must  therefore  be  valid.  Various  factors  can  affect  the  inter- 
pretation,  including  extemal  factors,  as  we  have  seen.  Sufficient  evidence  allows 
a conclusion  about  overall  test  quality  - its  validity.  It  starts  as  local  affair,  with 
repeated  use  of  a test  for  one  purpose  only,  and  ultimately  one  can  argue  that 
validity  becomes  a property  of  the  test,  i.e.  that  it  tests  what  it  purports  to  test;  that 
it  tests  a property  that  exists  and  can  be  measured. 


3.  Validation 

Validity  can  only  be  accessed  through  validation.  Validity,  in  Messick's  (1989) 
terms,  remains  an  abstract  concept,  and  validation  is  the  process  of  operationaliz- 
ing  the  construct  of  validity.  It  is  an  activity:  the  collection  of  all  possible  test- 
related  activities  from  multiple  sources.  The  validation  process  therefore  involves 
the  accumulation  of  evidence  to  support  the  proposed  test  score  interpretations 
and  uses  (Lane,  1999:  1).  The  process  is  based  on  Kane's  (1992)  systematic  ap- 
proach  to  thinking  through  the  process  of  validation.  Kane  sees  validation  as  the 
construction  of  "an  interpretative  argument";  a number  of  inferences  following 
each  other,  ending  in  decisions  made  about  a candidate. 

Davies  and  Elder  (2005:  804)  point  out  that  it  is  not  easy  to  operationalize 
Messick's  (1989)  and  Bachman's  (1990)  intricate  conception  of  validity.  McNama- 
ra  and  Roever  (2006: 33)  also  refer  to  a "decade  and  more  of  grappling"  with  this 
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complex  validity  framework.  Bachman  (2005:  267),  in  an  attempt  to  make  test 
validation  a manageable  process,  suggests  the  following  procedure: 

• Articulating  a validation  argument,  which  provides  the  logical  framework 
linking  test  performance  to  an  intended  interpretation  and  use. 

• Collecting  relevant  evidence  in  support  of  the  intended  interpretation 
and  use. 

Evidence  collected  may  include  what  Davies  and  Elder  (2005: 798)  call  the  "usual 
suspects"  of  content,  construct  and  criterion  validity,  as  well  as  reliability.  But 
additional  sources  of  validity  evidence  are  also  allowed  (Davies  & Elder,  2005: 
801),  mostly  as  part  of  the  social  dimension  of  testing  (consequential  validity 
interpreted  broadly),  such  as  student  feedback,  test  consequences,  ethics,  social 
responsibility,  washback,  affect  and  impact  of  test  scores.  But  there  is  not  always 
a prindpled  way  of  combining  all  the  elements  that  can  be  regarded  as  validation 
evidence.  Fulcher  and  Davidson  (2007: 18)  thus  speak  of  a pragmatic  approach  to 
validity;  an  approach  that  "best  explains  the  facts  available". 

The  validation  process  involves  the  development  of  a coherent  validity  argu- 
ment  for  and  against  proposed  test  score  interpretations  and  uses.  It  takes  the 
form  of  claims  or  hypotheses  (with  implied  counterclaims)  plus  relevant  evi- 
dence.  But  we  must  also  examine  potential  threats  to  the  validity  of  score  inter- 
pretation.  Kane,  Crooks  and  Cohen  (1999: 15)  point  out  that  "the  most  attention 
should  be  given  to  the  weakest  part  of  the  interpretative  argument  because  the 
overall  argument  is  only  as  strong  as  its  weakest  link".  Validation  is  therefore  as 
much  a process  of  raising  doubts  as  of  positive  assertion. 

What  constitutes  an  adequate  argument?  Fulcher  and  Davidson  (2007: 20) 
suggest  the  following  basic  criteria: 

• Simplidty:  explain  the  facts  in  as  simple  a manner  as  possible. 

• Coherence:  an  argumen  t must  be  in  keeping  with  what  we  already  know. 

• Testability:  the  argument  must  allow  us  to  make  predictions  about  future 
actions  or  relationships  between  variables  that  we  could  test. 

• Comprehensiveness:  as  little  as  possible  must  be  lef  t unexplained. 

We  now  illustrate  an  approach  to  test  validation  by  analysing  the  January  2007 
TAG  test  administration  and  results.  As  the  administration  of  the  test,  with  the 
interpretation  and  use  of  the  results,  is  an  expensive  and  important  exercise  for  its 
stakeholders  (university  management,  students,  parents  and  lecturers),  the  valid- 
ity  of  the  test  is  of  major  importance.  Davies  and  Elder  (2005: 802-3)  report  that 
relatively  few  comprehensive  validation  studies  have  been  undertaken,  and  this 
article  is  an  attempt  to  make  a contribution  in  this  regard. 
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4.  TAG  test:  Validation  claims  and  relevant  evidence 

The  TAG  test 


The  Toets  van  Akademiese  Geletterdheidsvlakke 1 (Test  of  Academic  Literacy 
Levels),  or  TAG,  was  administered  to  all  first-year  students  at  the  Potchef- 
stroom  campus  at  the  beginning  of  2007.  It  was  aimed  at  establishing  whether 
these  students  possessed  the  necessary  academic  literacy  skills  to  succeed 
in  their  content  subjects.  It  was  a medium  to  high  stake  test,  as  students 
who  failed  had  to  enrol  for  a course  in  Academic  Literacy,  for  which  par- 
ents  must  then  pay  an  extra  fee.  TAG  is  a short  test  of  55  minutes,  in  multi- 
ple-choice  format.  The  2007  test,  in  Afrikaans,  contained  63  items,  and  was 
administered  to  2773  students. 

The  test  content  was  based  on  a number  of  components  that  make  up  the 
construct  of  academic  literacy.  These  are  described  in  Van  Dyk  and  Weideman 
(2004)  and  Weideman  (2007).  It  is  assumed  that  these  components,  taken  together, 
constitute  the  construct  'academic  literacy'.  We  accept  this  definition  of  the  con- 
struct  as  valid  for  the  purposes  of  this  a posteriori  analysis. 

The  test  was  divided  into  sections  that  tested  the  following:  Placing  five  scram- 
bled  sentences  into  the  correct  sequence;  interpreting  a graph  (a  histogram); 
answering  comprehension  questions  on  a reading  passage;  deciding  which  phrase 
or  sentence  has  been  left  out  in  a text;  defining  academic  vocabulary  items;  iden- 
tifying  text  types;  and  deciding  where  a word  had  been  left  out  in  a text,  and 
which  word  had  been  left  out. 

Each  inference  in  a validity  argument  is  based  on  an  as- 
sumption  or  claim  that  requires  support  (Bachman,  2005: 264; 

Lane,  1999: 1;  Chapelle,  1999: 259).  In  our  validation  study  of 
the  TAG  test,  we  constructed  a number  of  claims  and  collected 
evidence  to  support  these  claims. 


Validation  evidence 

Claim  1 : The  test  is  reliable  and  provides  a consistent  measure, 
with  small  variance  the  result  of  measurement  error. 

A completely  reliable  test  implies  that  tests  scores  are  free  from 
errors  and  can  be  depended  on  for  making  decisions  about 
placement  or  admission.  The  use  of  intemal  consistency  coef- 
ficients  to  estimate  the  extent  of  the  reliability  of  objective- 
format  tests  is  the  industry  standard.  Reliability  coefficients  do  not  provide  evi- 
dence  of  test  quality  as  such:  the  estimated  reliability  is  "not  a feature  of  the  test, 
but  rather  of  a particular  administration  of  the  test  to  a given  group  of  examinees" 
(Weir,  2005: 30).  The  internal  consistency  for  each  section  and  for  the  whole  test 
was  determined  by  calculating  the  Cronbach  alpha  coefficients.  The  results  are 
displayed  in  Table  1. 


The  TAG  test  construct  and 
results  of  numerous 
administrations  of  different 
versions  of  the  test  have 
been  discussed  in  a number 
of  publications,  such  as  Van 
Dyk  and  Weideman  (2004), 
Van  der  SLik  and  Weideman 
(2005),  Weideman  (2005), 
Weideman  (2006a  & b)  and 
Weideman  (2007). 


Table  1: 

Cronbach  alpha  coefficients 

Alpha 

No.  of  items 

Section  1 

0,84 

5 

Section  2 

0,64 

7 

Section  3 

0,70 

22 

Section  4 

0,62 

9 

Section  5 

0,67 

5 

Section  6 

0,89 

15 

TAG 

0,88 

63 
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One  should  bear  in  mind  that  the  alpha  coefficient  is  a func- 
tion  of  the  number  of  items  it  is  based  on.  The  reliability  coeffi- 
cients  for  Sections  2, 3, 4,  and  5 are  below  the  generally  accepted 
norm  of  0,8  (Weir,  2005: 29),  while  the  alpha  for  the  test  as  a whole 
is  very  good  at  0,88. 

Claim  2:  The  general  ability  ofcandidates  matches  the  general 
level  ofdifficulty  oftest  items. 

An  item-response  analysis  was  performed,  using  the  FACETS  com- 
puter  program  (Facets  for  Windows  Version  No.  3.61.0).  A two- 
facet  Rasch  model  was  fitted  to  data  of  the  63  items  scores  of  the 
TAG  for  the  2773  students.  In  order  to  validate  this  model,  the 
following  diagnostic  methods  were  applied  (Hambleton,  Swami- 
nathan  & Rogers,  1991),  using  the  FACETS  program  (Facets  for 
Windows  Version  No.  3.61.0)  and  STATISTICA  (StatSoft,  Inc.,  2006): 

• A plot  of  the  student  abilities  of  the  more  difficult  items 
(upper  half  of  items  on  the  measurement  scale)  vs.  those  of 
the  easier  items  (remainder  of  items)  depicted  most  of  the 
points  to  lie  underneath  the  line  of  equality.  This  means 
that  the  model  based  on  the  more  difficult  items  predicts  a 
lower  ability  for  a student  than  a model  based  on  the  easier 
items. 

• A plot  of  the  student  abilities  of  the  odd-numbered  items 
vs.  those  of  the  even-numbered  displays  points  that  are 
evenly  distributed  along  the  equality  line,  with  high  cor- 
relation  (0,83),  which  means  that  a student's  abilities  are 
similar  for  two  sets  of  items  selected  from  the  test. 

• A plot  of  item  difficulties  for  higher  ability  students  vs. 
those  for  lower  ability  students:  with  a high  correlation 
(0,88)  and  even  distribution  of  the  points  along  the  equality 
line,  it  seems  that  the  difficulties  of  items  are  predicted 
similarly  for  the  model  fitted  on  the  low  and  high  ability 
groups  of  students. 

• Comparison  of  the  distributions  of  the  standardised  resid- 
uals  for  the  data  with  that  of  data  simulated  from  the  fitted 
model:  since  the  histograms  for  the  two  distributions  are 
very  similar,  the  model  fitted  simulated  data  in  the  same 
manner  as  that  of  the  original  data. 


Figure  1:  Item-ability  map 


We  therefore  conclude  that,  on  the  whole,  there  was  an  appropri- 
ate  fit  of  the  model  on  the  data. 
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The  item-ability  map  in  Figure  1 displays  the  distributions  of  the  students' 
abilities  and  the  TAG  item  difficulties,  both  relative  to  a logit-scale  measure.  This 
measure  varies  from  +5  at  the  top  to  -5  at  the  bottom,  the  larger  values  indicating 
better  student  abilities  and  more  difficult  items,  while  lower  values  indicate  poorer 
student  abilities  and  easier  items.  The  map  provides  estimates  of  person  ability 
and  item  difficulty.  These  are  expressed  in  terms  of  the  relation  between  the 
ability  of  individual  candidates  and  their  relative  chances  of  giving  a correct 
response  to  items  of  given  difficulty;  the  chances  being  expressed  as  logits  (Mc- 
Namara,  1996: 200).  This  map  allows  comparison  of  candidate  ability  and  item 
difficulty. 

From  this  display  it  is  clear  that  no  extreme  difficulties  occurred,  while  only  a 
very  few  students  had  extreme  abilities  outside  the  limits  -3  and  3.  There  was  no 
significant  mismatch;  the  ability  of  the  candidature  was  at  the  general  level  of 
difficulty  of  the  items,  and  there  was  a good  fit  between  student  ability  and  item 
difficulty. 

Claim  3:  Infit  mean  square  values  of  test  items  fall  within  an  acceptable  range. 

By  means  of  item-response  analysis,  a Rasch  model  that  summarises  the  observed 
patterning  throughout  the  set  of  relations  between  candidates  and  test  items  was 
fitted.  Here  we  wish  to  consider  the  extent  to  which  the  pattern  of  responses 
observed  for  individual  items  conforms  to  and  reinforces  the  general  pattern  in 
the  model,  or  goes  against  it  (McNamara,  1996: 169).  If  the  pattern  for  the  individ- 
ual  items,  allowing  for  normal  variability,  fits  the  overall  pattern,  the  items  show 
appropriate  'fit'.  If  not,  they  are  'misfitting'  or  'overfitting'  items  (McNamara,  1996: 
169-175).  The  fit  statistics  in  Table  2 give  tlie  difficulty  levels  of  items  as  measured 
on  the  logit-scale,  ordered  from  the  most  difficult  item  (no.  21  with  measure  1,77) 
to  the  easiest  one  (item  1 with  measure  -2,2  7),  together  with  the  fit  statistic  'infit 
mean  square'. 

McNamara  (1996: 172)  points  out  that  infit  statistics  are  informative  as  they 
focus  on  the  degree  of  fit  in  the  most  typical  observations  in  the  model.  He  states 
that  infit  mean  square  values  have  an  expected  value  of  1;  individual  values  will 
be  above  or  below  this  according  to  whether  the  observed  values  show  greater 
variation  (resulting  in  values  greater  than  1)  or  less  variation  (resulting  in  values 
less  than  1)  (McNamara,  1996: 172).  McNamara  (1996: 173)  suggests  that  values  in 
the  range  of  0,75  to  1,3  are  acceptable.  Values  greater  than  1,3  show  significant 
misfit,  i.e.  lack  of  predictability,  while  values  below  0,75  show  significant  overfit. 

Since  the  infit  mean  squares  have  values  that  vary  between  0,97  and  1,04,  all 
items  seem  to  be  in  accordance  with  the  fitted  Rasch  model. 
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Table  2:  Fit  statistics 


Observed  % correct 

Measure 

Infit  Mean  square 

TAG  item 

19.8 

1.77 

1.02 

21 

25.5 

1.39 

0.99 

46 

30.9 

1.08 

1.02 

45 

32.3 

1 

1 

61 

32.6 

0.99 

1.01 

50 

34.9 

0.87 

1.01 

9 

37.3 

0.74 

0.99 

20 

37.3 

0.74 

0.99 

32 

37.5 

0.73 

1 

63 

38.2 

0.7 

0.97 

17 

38.2 

0.7 

0.99 

41 

38.7 

0.68 

0.99 

60 

39.9 

0.62 

1 

16 

40.3 

0.6 

1.01 

37 

41.7 

0.53 

1 

23 

41.8 

0.53 

1 

26 

41.8 

0.52 

1.02 

56 

41.9 

0.52 

0.97 

49 

42.8 

0.47 

1 

57 

43.4 

0.45 

1 

62 

43.5 

0.44 

0.98 

42 

43.5 

0.44 

0.99 

15 

45.1 

0.37 

0.98 

35 

46.7 

0.29 

0.97 

34 

47.9 

0.23 

1.02 

8 

50.3 

0.12 

1.04 

3 

50.5 

0.11 

1.01 

40 

50.6 

0.11 

1 

59 

50.6 

0.11 

1.01 

27 

51.8 

0.05 

0.99 

18 

52.3 

0.03 

0.98 

19 

52.7 

0.01 

1 

2 

53.0 

-0.01 

1 

11 

53.0 

-0.01 

0.99 

24 

53.3 

-0.02 

1 

47 

53.5 

-0.03 

0.99 

4 
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Observed  % correct 

Measure 

Infit  Mean  square 

TAG  item 

53.6 

-0.03 

1 

58 

54.4 

-0.07 

0.99 

31 

54.6 

-0.08 

1.02 

44 

54.9 

-0.09 

1.01 

10 

57.2 

-0.2 

1 

22 

58.6 

-0.27 

1.02 

14 

59.5 

-0.31 

1.01 

43 

61.0 

-0.39 

1 

25 

62.1 

-0.44 

1 

55 

62.1 

-0.44 

0.99 

12 

63.0 

-0.48 

0.99 

54 

63.1 

-0.49 

1 

30 

63.4 

-0.51 

1.02 

51 

65.6 

-0.61 

1.01 

48 

65.8 

-0.62 

0.99 

29 

70.0 

-0.85 

1 

53 

70.6 

-0.88 

1.01 

5 

71.2 

-0.91 

1.01 

28 

71.5 

-0.93 

0.98 

13 

72.7 

-1 

0.99 

52 

74.6 

-1.11 

1 

7 

75.3 

-1.15 

1.02 

39 

75.7 

-1.18 

1.02 

6 

83.1 

-1.69 

1 

36 

88.5 

-2.18 

0.99 

38 

89.4 

-2.27 

1 

1 

Claim  4:  The  internal  correlations  of  the  different  test  sections  satisfy  specific 
criteria. 

Bachman  (1990: 258)  states  that  patterns  of  correlations  among  item  scores  and 
overall  test  scores  provide  evidence  of  construct  validity.  Alderson  et  al.  (2005: 
184)  indicate  that  an  internal  correlation  study  can  be  used  in  order  to  examine 
this.  They  point  out  that  the  reason  for  having  different  test  sections  is  that  they 
all  measure  something  different  and  therefore  contribute  to  the  overall  picture  of 
the  attribute.  They  expect  these  correlations  to  be  fairly  low  - possibly  in  the 
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Table  3:  Correlations  order  0,3  to  0,5.  If  two  sections  (components) 


Section 

1 

2 

3 

4 

5 

6 Total 

correlate  very  highly  with  each  other  (e.g. 

1 

0,294 

0,9),  one  might  wonder  whether  the  two  sec- 

2 

0,207 

0,387 

tions  are  testing  different  attributes,  or 

3 

0,297 

0,378 

0,514 

whether  they  are  testing  essentially  the  same 

4 

0,214 

0,312 

0,495 

0,573 

thing.  The  correlations  between  each  sec- 

5 

0,141 

0,161 

0,304 

0,358 

0,441 

tion  and  the  whole  test,  on  the  other  hand. 

6 

0,146 

0,195 

0,282 

0,388 

0,391 

0,406 

might  be  expected  to  be  higher  - possibly 

Total 

0,447 

0,533 

0,763 

0,697 

0,557 

0,737 

around  0,7  or  more  - since  the  overall  score 

is  taken  to  be  a more  general  measure  of  the 
attribute  than  each  individual  section  score. 
Alderson  et  al.  (2005: 184)  add:  "Obviously  if  the  individual  component  score  is 
included  in  the  total  score  for  the  test,  then  the  correlation  will  be  partly  between 
the  test  component  and  itself,  which  will  artificially  inflate  the  correlation.  For 
this  reason  it  is  common  in  internal  correlation  studies  to  correlate  the  test  com- 
ponents  with  the  test  total  minus  the  component  in  question".  Three  different 
types  of  correlation  coefficients  can  be  identified,  each  with  its  own  criterion: 

• The  correlation  coefficients  between  each  pair  of  subtests  (Cl).  These  cor- 
relations  should  be  fairly  low,  from  0,3  to  0,5  (cf.  Hughes,  1989: 160;  Alder- 
son  et  al.,  2005: 184;  Ito,  2005). 

• The  correlation  coeffidents  between  each  subtest  and  whole  test  (C2).  These 
correlations  should  be  0,7  and  more  (cf.  Alderson  et  al.,  2005: 184;  Ito,  2005). 

• The  correlation  coefficients  between  each  subtest  and  the  whole  test  mi- 
nus  the  subtest  (C3).  These  should  be  lower  than  those  between  each  subtest 
and  the  whole  test,  i.e.  C2  > C3  (cf.  Ito,  2005). 

The  correlational  pattern  that  results  is  indicated  in  Table  3.  The  table  indicates 
the  following: 

• C1  (shaded  areas):  only  eight  of  the  fifteen  correlations  meet  the  criterion, 
with  seven  lower  than  0,3. 

• C2  (last  row):  only  three  of  the  six  correlations  meet  the  criterion. 

• C3  (last  column):  all  correlations  meet  the  criterion. 

It  must  be  noted  that  one  weakness  inherent  in  the  correlational  approach  to 
construct  validation  is  that  it  only  evaluates  the  relevance  of  those  performance 
criteria  that  are  already  included,  and  that  it  cannot  identify  others  that  are  rele- 
vant  to  the  construct  but  which  have  been  omitted  (Moritoshi,  2002: 11) 

Claim  5:  Each  sedion  of  the  TAG  test  displays  construct  validity. 

Bachman  (1990: 259)  indicates  that  factor  analysis  is  extensively  employed  in  con- 
struct  validation  studies.  The  construct  validity  of  each  section  of  the  test  can  be 
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verified  by  means  of  principal  component 
analysis,  a factor  analytic  model  that  reduc- 
es  data  and  extracts  the  principal  compo- 
nents  or  factors  that  underlie  the  construct 
being  assessed.  The  results  obtained  by 
means  of  STATISTICA  (StatSoft,  Inc.,  2006) 
are  displayed  in  Table  4. 

Only  Sections  1 and  5 formed  one  con- 
struct,  while  sections  2 and  6 can  be  split  up 
into  two  constructs.  In  this  regard,  a princi- 
pal  factor  analysis  with  an  oblique  (OBLIM- 
IN)  rotation  was  performed.  For  section  2 
the  resultant  factor  pattem  gives  items  6-9, 

12  as  first  sub-factor,  while  items  10  and  11 
belong  to  the  second  sub-factor,  with  a cor- 
relation  of  -0,57  between  the  sub-factors.  In 
the  case  of  section  6,  the  sub-factors  are 
formed  by  items  56  to  63  and  49  to  55  respec- 
tively,  with  a correlation  of  -0,64  between 
sub-factors.  Sections  3 and  4 are  not  con- 
struct  valid.  To  be  construct  valid,  as  few  as 
possible  factors  that  explain  the  maximum 
percentage  of  variance  are  required,  with 
communalities  as  high  as  possible.  (In  Sec- 
tion  4,  only  37%  of  the  variation  is  explained 
by  the  two  factors.) 

Claim  6:  Thefirst  principal  component  dominates  the  whole  test. 

In  Figure  2,  the  eigenvalues  for  each  principal  component  are  plotted  in  a simple 
scree  plot  line  (cf.  Cattell,  1966)  in  their  sizes  as  a diminishing  series.  The  percent- 
age  variance  explained  by  the  first  component  was  13,5%  relative  to  6,2%,  3,9%, 
3,0%  of  the  second,  third  and  fourth  components  respectively.  The  first  compo- 
nent  is  therefore  not  as  dominant  as  one  should  ideally  wish. 

Claim  7:  Test  takers  werefam iliar  with  the  demands  of  the  test. 

A questionnaire  aimed  at  establishinghow  first-year  students  experienced  the  test 
was  completed  by  a group  of  754  students  from  eight  faculties  during  the  first  week 
of  classes.  Because  of  practical  constraints,  this  group  was  not  randomly  selected, 
but  formed  an  availability  sample.  The  findings  cannot  be  generalised  to  the  whole 
population,  but  provide  an  indication  of  how  students  felt  about  the  whole  test 
procedure.  Claims  7 to  10  are  based  on  the  findings  of  this  questionnaire. 
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Table  4:  Factor  analysis  results 

Section  Number  of  Percentage  Communalities 
components  variance 
explained 

1 
2 

3 

4 

5 

6 


10 
9 
8 
7 
6 

4 
3 
2 
1 
0 

0 2 4 6 8 10  12  14  lt>  18  20 

Number  cf  eigenvoiues 

Figure  2:  Scree  plot 


1 

61 

0.18  - 0.81 

2 

71 

0.25  - 0.68 

6 

39 

0.18  - 0.70 

2 

37 

0.20  - 0.59 

1 

43 

0.37  - 0.48 

2 

50 

0.15-0.66 

\ 


1 50 
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No  response 


Other 


Table  5:  Where  did  you  first  hear  about  the  test? 


From  friends 


When  I enrolled 


In  the  first-year  guide 
From  the  residence  committee 


Percentage 


20 


42 


13 


16 


7 


2 


Weir  (2005:  54)  points  out  that  candidates  should 
be  familiar  with  the  task  type  before  sitting  the  test 
proper,  as  the  degree  of  the  candidate's  familiarity 
with  the  demands  of  a test  may  affect  the  way  the 
task  is  dealt  with.  He  states  that  specimen  past  pa- 
pers  and  clear  specifications  should  help  difficul- 
ties  in  this  respect,  and  that  an  exemplification  of 
tasks  and  procedures  should  be  readily  available. 


Students  were  informed  about  the  TAG  test 
when  they  enrolled  at  the  university,  and  full  in- 


formation  was  provided  in  a first-year  guide  that  was  sent  to  them  in  the  November 
preceding  their  arrival  towards  the  end  of  January.  No  information  was  provided 
on  the  university  website.  Table  5 indicates  where  students  first  heard  about  the 
test 

Most  of  these  students  read  about  the  test  in  the  first-year  guide.  The  guide 
informed  students  that  they  could  look  at  a specimen  test  on  the  website  of  the 
University  of  Pretoria.  Only  11  percent  looked  at  this  example  test.  Of  these,  only 
7 percent  did  the  practice  test.  Seventy-five  percent  who  did  not  look  at  the 
specimen  test  indicated  that  they  would  have  liked  to  do  so.  It  therefore  seems  as 
if  the  test  is  not  as  transparent  as  it  could  be  at  the  Potchefstroom  campus,  and  that 
much  more  could  be  done  to  inform  students  about  the  test  and  its  format. 

Claim  8:  The  circumstances  under  which  the  test  is  administered  are  conducive  to 


valid  scores  being  obtained. 


The  test  was  written  on  the  fourth  day  after  students'  arrival  on  campus.  From  the 
second  day,  they  are  subjected  to  an  Orientation  and  Information  week  under  the 
supervision  of  the  residence  committees.  This  involves  a full  programme,  and 
students  are  kept  busy  the  whole  day  (and  part  of  the  night!). 

We  asked  students  whether  they  felt  they  could  deliver  their  best  perform- 
ance  in  the  test:  65  percent  indicated  that  they  could  not.  The  reasons  they  indi- 
cated  were  as  follows:  tired  (40%),  sleepy  (21%),  stressed  (25%),  ill  (7%)  and  other 
reasons  (7%).  Sixty-three  percent  of  the  students  went  to  bed  after  midnight, 
while  57  percent  reported  that  they  had  to  get  up  before  6:00  on  the  morning  of 
the  test.  It  seems  as  if  the  circumstances  in  which  the  test  is  administered  are  not 
ideal,  and  these  could  affect  the  validity  of  the  test  results. 

Claim  9:  Students  experience  the  test  as  relevant  to  their  studies. 

Forty-five  percent  of  the  students  had  a good  idea  of  the  purpose  of  the  test;  thirty- 
four  percent  had  a vague  idea,  while  21  percent  did  not  know  what  the  purpose 
was.  The  test  results  were  indicated  in  terms  of  codes  ranging  from  1 to  5,  indicat- 
ing  to  what  extent  they  were  at  risk  in  their  studies.  Students  whose  results  fell 
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within  the  codes  1 to  3 must  enrol  for  the  course  in  Academic  Literacy.  Twenty- 
nine  percent  of  the  respondents  had  to  take  the  course.  Sixty-nine  percent  of  those 
who  had  to  take  the  course  declared  that  they  would  do  so  under  protest. 

Claim  10:  Students  found  the  test  experience  agreeable. 

Eleven  percent  of  the  students  said  they  found  all  the  questions  clear,  68%  knew 
what  they  had  to  do  at  most  questions,  19%  at  some  of  the  questions,  while  1%  did 
not  understand  any  question  (1%  gave  no  reply).  We  also  asked  the  students  what 
they  thought  about  the  length  of  the  test.  Fourteen  percent  could  finish  the  test, 
39%  could  finish  but  had  to  work  fast,  while  47%  reported  that  they  could  not 
finish  the  test.  This  is  somewhat  disconcerting,  as  most  responses  should  ideally 
be  in  the  middle  category.  Fifty-eight  percent  felt  that  the  test  was  too  long  to 
finish  within  the  allocated  time. 


5.  Condusion 

A problem  in  the  current  conception  of  validation  studies  is  the  balance  between 
theoretical  rigour  and  manageability  - this  remains  a challenge  for  validation 
research.  As  a result,  a pragmatic  stance  is  often  adopted,  as  was  the  case  here.  The 
framework  employed  in  this  validation  study  includes  statistical  procedures, 
based  on  both  classical  and  item-response  theory,  as  well  as  a sodal  dimension,  in 
the  form  of  student  feedback.  A variety  of  evidence  was  collected,  providing  a 
profile  of  the  test  results  and  its  administration.  It  is  obvious  that  validity  is  a 
multifaceted  concept,  and  that  many  factors  together  play  a role  in  the  validation 
of  a test.  Each  claim  that  is  formulated  contributes  to  an  aspect  of  the  validity  of 
the  test.  The  aim  of  the  article  was  to  illustrate  a method  of  doing  a validation 
study,  and  it  is  clear  that  the  conclusions  made  depend  on  a judgement  and 
interpretation  of  the  results  obtained. 

It  must  be  stressed  that  there  is  no  such  thing  as  a perfect  test,  as  is  no  test 
situation.  This  is  why  provision  is  usually  made  for  the  misclassification  of  candi- 
dates  when  tests  results  are  analysed.  But  the  TAG  test  investigated  here  performs 
very  well.  It  is  used  for  a specific,  clearly  defined  purpose.  Its  reliability  is  good, 
and  there  is  a good  fit  between  student  ability  and  item  difficulty.  The  internal 
correlations  are  probably  as  good  as  can  be  expected.  More  than  one  underlying 
trait  (or  factor)  was  extracted,  and  the  first  was  not  as  dominant  as  expected.  This 
may  be  due  to  the  fact  that  academic  literacy  is  a rich  and  multidimensional 
construct  (cf.  Van  der  Slik  & Weideman,  2005).  It  is  also  clear  that  much  more  can 
be  done  to  improve  the  administration  of  the  test,  such  as  informing  students 
better,  explaining  the  relevance  of  the  test,  and  ensuring  that  students  can  deliver 
their  best  performance  in  the  test.  The  problems  students  had  with  the  length  of 
the  test  also  warrant  further  investigation. 
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We  investigated  the  validity  of  only  one  administration  of  the  TAG  test  here.  If 
it  proves  to  be  valid  in  most  respects  over  number  of  administrations,  and  re- 
mains  to  be  used  for  its  specific  purpose,  the  test  itself  can  come  to  be  regarded  as 
a valid  instrument,  in  terms  of  the  traditional  interpretation  of  validity.  Valida- 
tion  thus  remains  an  ongoing  process. 
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Van  bevreemdende  diskoers  tot  toegangsportaal:  e-Leer  as 
aanvullingtot  'n  akademiese  geletterdheidskursus 


T.J.  van  Dyk,  L.  van  Dyk,  H.C.  Blanckenberg  en  J.  Blanckenberg 
Universiteit  Stellenbosch 


From  alienating  discourse  to  access  discourse:  e-Learning  as  supplement  to 
an  academic  literacy  course 

Low  levels  ofskill  in  the  language  of  teaching  and  learning  is  widely  considered 
one  ofthe  main  reasonsfor  the  lack  of  academic  success  among  South  African  (and 
intemational)  undergraduate  students.  lnfact,  students  are  also  mcreasingly 
experiencing  academic  discourse  in  their  mother  tongue  as  a "foreign  language".  It 
is  an  unsettling  trend  which  is  confirmed  by  local  and  international  literature, 
and  can  possibly  be  attributed  to  the  language  curriculum  in  secondary  education 
that  does  not  adequately  prepare  studentsfor  the  higher  order  language-thinking 
skills  that  they  needfor  study  at  university.  During  the  pastfew  years  various 
local  and  international  tertiary  education  institutions  have  started  implementing 
academic  literacy  programmes  for  all  first-year  students  to  address  this  under- 
preparedness.  Two  problems,  however,  have  a significant  influence  on  the  effedive 
teaching  ofthese  language-thinking  skills.  Firstly,  faculties  can  hardly  sacrifice 
two  contact  sessions  per  week  to  these  types  ofsupport  courses  because  oftheir  own 
overfull  programmes.  Secondly,  there  are  only  a limited  number  oflanguage 
praditioners  available for  an  increasing  number  ofstudents.  As  a possible 
solution,  theSU  Language  Centre  has  launched  an  e-learning project  within 
which  a supplementary  course-related  e-learning  modulefor  an  existingfirst-year 
reading  course  has  been  developed.  From  the  quantitative  and  qualitative  research 
results  insightful  conclusions  can  be  drawn.  Examples  are:  (i)  the  extent  to  which 
e-learning can  supplement  lecture  time,  while  simultaneously  empowering 
students  academically;  and  (ii)  the  effediveness  of  including  in  the  curriculum  e- 
learning  task  types  that  facilitate  the  acquisition  ofcertain  higher  order  language- 
thinking  skills. 


1.  Inleiding 

Universiteite  ervaar  deesdae  enersyds  druk  om  student-dosent-ratio's  te  verhoog 
aangesien  daar  beperkte  finansiële  hulpbronne  beskikbaar  is,  maar  andersyds  is 
daar  ook  druk  om  meer  steun  en  onderrigtyd  te  wy  aan  die  ontwikkeling  van 
studente  se  kennis  en  vaardighede  (onder  andere  met  betrekking  tot  akademiese 
geletterdheid).  'n  Balans  moet  derhalwe  gevind  word  tussen  die  finansiële  le- 
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wensvatbaarheid  van  universiteite  en  hul  verantwoordelikheid  om  die  nodige 
hulpbronne  hiervoor  beskikbaar  te  stel.  In  hierdie  opsigis  'n  loodsprojekby  die 
Universiteit  Stellenbosch  (US)  van  stapel  gestuur  om  te  bepaaal  of  kontakonder- 
rig,  binne  konteks  van  'n  akademiese  geletterdheidsprogram,  met  e-leer  aangevul 
kan  word  en  indien  wel,  in  welke  mate  en  in  welke  vorm. 

Daar  word  in  hierdie  artikel  eerstens  'n  breë  agtergrond  geskets  waarteen  die 
probleem  beskou  moet  word.  Faktore  wat  aanleiding  gegee  het  tot  die  noodsaak- 
likheid  om  alternatiewe  tot  kontakonderrig  te  ondersoek,  word  daama  bespreek. 
Die  potensiaal  van  e-leer  as  so  'n  altematief  word  belig  waama  die  aanpak  van  die 
loodsprojek,  om  die  haalbaarheid  hiervan  te  ondersoek,  beskryf  word.  Die  res 
van  die  artikel  word  gewy  aan  die  bevindinge  en  gevolgtrekkings  wat  spruit  uit 
kwalitatiewe  en  kwantitatiewe  terugvoer  van  studente  wat  deelgeneem  het  aan 
die  projek. 


2.  Agtergrond 

Op  grond  van  data  van  die  Suid-Afrikaanse  Nasionale  Departement  van  Onder- 
wys  wys  Mamaila  (2001 :1)  daarop  dat  25%  van  eerstejaarstudente  wat  vir  univer- 
siteitstudie  ingeskryf  is,  reeds  vóór  die  voltooiing  van  die  eerste  studiejaar  die 
handdoek  ingooi.  Die  situasie  bly  kommerwekkend  in  die  daaropvolgende  jare 
van  die  voorgraadse  program.  Tussen  30%  en  40%  van  eerstejaarstudente  voltooi 
byvoorbeeld  nie  hul  voorgraadse  studies  in  die  aanbevole  tydperk  nie.  Deur- 
vloeikoerse  aan  hoëronderwysinstellings  is  dus  laag  en  verteenwoordig'n  finan- 
siële  verlies  van  bykans  Rl,8  miljard  per  jaar. 

Daar  is  verskeie  redes  vir  hierdie  situasie,  maar  lae  vlakke  van  vaardigheid  in 
die  taal  van  onderrig  en  leer  word  as  een  van  die  hoofredes  vir  'n  gebrek  aan 
akademiese  sukses  onder  Suid-Afrikaanse  voorgraadse  studente  beskou  - selfs 
onder  diegene  met  groot  akademiese  potensiaal  (Blacquiére,  1989;  Leibowitz,  2001; 
Macdonald,  1990;  Perkins,  1991;  Pretorius,  1995;  Van  Rensburg  & Weideman,  2002; 
Vorster  & Reagan,  1990).  Navorsing  deur  onafhanklike  onderwysspesialiste  beves- 
tig  verder  'n  ingrypende  daling  in  die  (akademiese)  taalvaardigheidsvlak  van 
studente  wat  vir  tersiêre  studie  inskryf  (Natal  Witness,  2004:8;  Rademeyer,  2005a:40). 
Dit  word  verder  bevestig  deur  die  uitslae  van  akademiese  geletterdheidstoetse 
wat  deesdae  by  verskeie  universiteite  as  plasings-  en  selfs  toegangsmeganismes 
gebruik  word.  'n  Vanselfsprekende  gevolg  van  die  genoemde  lae  akademiese 
geletterdheidsvlakke  is  dat  sodanige  studente  moeilik  met  voorgeskrewe  akade- 
miese  materiaal  omgaan. 

Die  toelating  van  studente  wat  ondervoorbereid  is  ten  opsigte  van  akade- 
miese  taalvaardigheid  (Zamel  & Spack,  1998),  blyk  nie  net  'n  plaaslike  verskynsel 
te  wees  nie  (Butler  & Van  Dyk,  2004:1).  Pierce  (2003)  doen  byvoorbeeld  verslag  oor 
navorsing  wat  daarop  dui  dat  slegs  6%  van  eerstejaarstudente  aan  universiteite  in 
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die  Verenigde  State  van  Amerika  (VSA)  gereed  is  om  selfstandig  uit  hulle  hand- 
boeke  te  leer.  Dié  toedrag  van  sake  verswak  selfs  verder,  aangesien  diegene  wat 
goed  kan  skryf  slegs  2%  van  die  totaal  uitmaak,  terwyl  minder  as  'n  derde  (31%) 
oor  voldoende  skryfvermoë  beskik.  'n  Studie  wat  McKenzie  en  Schweitzer  (2001) 
onder  Australiese  studente  uitgevoer  het,  bevestig  hierdie  bevindinge  en  beklem- 
toon  ook  dat  akademiese  taalvaardigheid  akademiese  sukses  beïnvloed. 

Verskeie  redes  word  plaaslik  vir  akademiese  mislukking  aangevoer  (waarvan 
daar  vir  doeleindes  van  hierdie  artikel  slegs  kortliks  na  twee  verwys  sal  word).  Die 
eerste  is  die  politieke  geskiedenis  van  apartheid  en  die  gevolglike  ongelyke  verde- 
ling  van  hulpbronne  in  die  Suid-Afrikaanse  onderwysstelsel,  wat  'n  groot  groep 
studente  negatief  geraak  het  (Butler  & Van  Dyk,  2004:1).  Swak  voorbereidheid  is 
egter  'n  verskynsel  wat  nie  meer  net  onder  histories  benadeelde  studente  te  be- 
speur  is  nie.  'n  Toenemende  aantal  tradisioneel  wit,  sogenaamd  bevoorregte  stu- 
dente  staar  ook  akademiese  mislukking  in  die  gesig.  Trouens,  ook  moedertaalstu- 
dente  ervaar  toenemend  die  akademiese  diskoers  in  hul  moedertaal  as  bevreemdend. 

'n  Verdere  rede  vir  lae  akademiese  geletterdheidsvlakke  kan  wees  dat  die  Suid- 
Afrikaanse  onderwysstelsel  in  die  verlede  sterk  leerplangedrewe  en  positivisties 
was.  Taal  is  gesien  as  'n  versameling  diskrete  elemente  wat  geleer  en  nie  verwerf 
hoef  te  word  nie.  Binne  hierdie  stelsel 

• is  kennis  as  universeel,  onveranderlik,  konteksvry  en  neutraal  beskou; 

• was  die  onderwyser  die  enigste  inligtingsbron,  en  is  sy/haar  sienings  as 
beste  kundige  nooitbevraagteken  nie; 

• was  persoonlike  ervaring  en  kultuur,  of  die  bekende  werklikheid,  die  enig- 
ste  geldige  verwysingsraamwerk; 

• het  die  klem  op  linkerbreinaktiwiteite  grootliks  tot  'n  liniêre  denkwyse 
aanleiding  gegee;  en 

• was  begrip  ongedifferensieerd,  met  die  gevolglike  ervaring  van  alle  in- 
ligting  as  ewe  belangrik. 

(Bencze,  2005;  Blanckenberg,  1999:34) 

Tans  is  onderwysers  vasgevang  tussen  die  positivistiese  patrone  van  die  verlede 
en  die  hedendaagse  konstruktivistiese  benadering  tot  onderrig  en  leer  (Blancken- 
berg,  1999:41).  Onderwysers  het  oor  die  algemeen  nog  nie  die  skuif  na  die  kon- 
struktivistiese  benadering,  wat  die  grondslag  van  nuwe  verwikkelinge  soos 
uitkomsgebaseerde  onderwys  (UBO)  vorm,  gemaak  nie.  Hiervoor  bestaan  daar 
etlike  ingewikkelde  redes  (wat  vir  die  doeleindes  van  hierdie  artikel  nie  bespreek 
sal  word  nie),  maar  waarvan  'n  hoofrede  moontlik  'n  gebrek  aan  opleiding  en 
hulpbronne  is  om  so  'n  skuif  moontlik  te  maak.  Op  die  oomblik  weifel  onderwys- 
ers  dus  tussen  die  ou  en  die  nuwe  benadering  tot  leer  en  onderrig. 

Rademeyer  (2005a:41)  stel  die  ironiese  moontlikheid  dat  studente  as  gevolg 
van  UBO  leerprobleme  in  die  tersiêre  leeromgewing  kan  ondervind  aangesien 


Ensovoort:  jaargang  1 1 , nommer  2,  2007 


157 


die  doel  daarvan  is  om  leerders  vir  die  werklike  lewe  (outentieke  situasies)  en  nie 
noodwendig  vir  universiteitstudie  voor  te  berei  nie.  Leerders  ontwikkel  dus  nie 
meer  noodwendig  die  vermoë  om  as  selfstandige,  strategiese,  besinnende  den- 
kers  op  te  tree  nie,  en  die  ontwikkeling  van  hierdie  vermoë  is  'n  basiese,  nood- 
saaklikheid  vir  enige  universiteitstudent.  Gevolglik  sukkel  universiteitstudente 
(veral  eerstejaars)  onder  andere  daarmee  om  taakopdragte,  toetsvrae  en  akade- 
miese  tekste  met  'n  redelik  gesofistikeerde  argument  en  abstrakte  woordeskat 
strategies  te  lees  en  te  verstaan.  Dit  is  dus  duidelik  dat  die  taalkurrikulum  in  die 
sekondêre  onderwys  leerders  nie  toereikend  voorberei  op  die  hoërorde-taal- 
denkvaardighede  wat  hulle  hiervoor  benodig  nie. 

Bostaande  situasie  laat  universiteite  met  geen  ander  keuse  nie  as  om  meganis- 
mes  in  werking  te  stel  om  ondervoorbereide  studente  te  ondersteun  deur  aandag 
te  skenk  aan  faktore  soos  lae  akademiese  geletterdheidsvlakke  wat  akademiese 
sukses  ná  inskrywingbeïnvloed  (Botha  & Cilliers,  1999:144).  Een  sodanige  mega- 
nisme  wat  die  Universiteit  Stellenbosch  (US)  besluit  het  om  te  implementeer,  is 
die  instelling  van  akademiese  geletterdheidskursusse  op  verskillende  vlakke  vir 
alle  eerstejaarstudente  na  gelang  van  die  risikovlak  (aldus  die  akademiese  gelet- 
terdheidstoetse)  van  die  studente.  Die  US  Taalsentrum  is  gevolglik  getaak  om  die 
verantwoordelikheid  vir  hierdie  kursusse  te  neem.  Nog  nie  alle  fakulteite  het  dit 
geïmplementeer  nie,  want  die  besluit  is  relatief  onlangs  eers  geneem. 


3.  Probleemstelling  en  voorgestelde  oplossing 

Die  infasering  van  kredietdraende  akademiese  geletterdheidskursusse  wat  stu- 
dente  onder  andere  bemagtig  om  op  ten  minste  'n  akademies  aanvaarbare  vlak  te 
lees  en  skryf,  bring  egter  'n  bepaalde  problematiekmee. 

Sekere  fakulteite  (Natuurwetenskappe  en  Ekonomiese  en  Bestuurswetenskap- 
pe)  se  eerstejaarsgetalle  is  byvoorbeeld  so  groot  dat  meer  as  een  dosent  in  een 
periodegleuf  sal  moet  klasgee,  of  dat  lesings  herhaal  sal  moet  word,  wat  logisties 
en  finansieel  gesproke  nie  haalbaar  is  nie.  Hierdie  verskynsel  gaan  toeneem  na- 
mate  alle  fakulteite  bedien  moet  word.  Verder  staan  fakulteite  oor  die  algemeen 
kwalik  twee  kontaksessies  per  week  af  aan  hierdie  soort  steunkursusse,  weens  eie 
oorvol  programme.  Gevolglik  kan  konsepte  bespreek  en  toegepas  word,  maar  tyd 
en  geleentheid  vir  uiters  noodsaaklike  vaslegging/inoefening  ontbreek. 

Die  Taalsentrum  moes  derhalwe  op  kreatiewe  maniere  met  oplossings  voren- 
dagkom.  'n  Moontlikheid  was  die  ontwikkeling  van  'n  aanvullende  kursusver- 
wante  e-leerprogram  wat  konsepte  en  vaardighede,  ingelei  en  gedemonstreer 
tydens  kontaklesings,  verder  inoefen  en  evalueer.  Sodoende  sou  ten  minste  die 
doseerlas  (alhoewel  nie  die  totale  werklas  nie)  van  dosente  verlig  kon  word. 

Die  volgende  drie  voordele  van  e-leer  lê  aan  die  hart  van  hierdie  moontlike 
oplossing: 


158 


Ensovoort:  jaarganc  1 1 , nommer  2,  2007 


• Alhoewel  die  ontwikkeling  van  e-leermateriaal  eerstens  geweldig  ar- 
beidsintensief  is,  is  een  van  die  grootste  voordele  die  potensiaal  daarvan 
om  na  baie  studente  uitgebrei  te  kan  word. 

• Omdat  akademiese  geletterdheidskursusse  ten  tweede  aan  so  'n  groot  ge- 
deelte  van  die  studentebevolking  gebied  word,  sou  dit  dus  sinvol  wees 
om  deur  'n  loodsstudie  te  bepaal  of  dit  prakties  uitvoerbaar  is  om  e-leerkom- 
ponente  ook  in  ander  modules  te  gebruik  (die  voordeel  van  die  ekonomie 
van  skaal  dus). 

• Derdens  was  Prensky  (2001)  die  eerste  persoon  om  na  hedendaagse  stu- 
dente  as  sogenaamde  digitale  outochtone  ("digital  natives")  te  verwys. 
Selfs  die  student  wat  in  'n  huis  grootword  sonder  elektrisiteit,  het  byvoor- 
beeld  dikwels  toegang  tot  'n  selfoon  - al  word  dit  met  'n  motorbattery 
gelaai.  Daar  sou  dus  verwag  kon  word  dat  studente  min  aanpassingspro- 
bleme  sal  ondervind  met  'n  aanvullende  e-leerprogram.  Meer  nog:  omdat 
studente  oor  die  algemeen  tuis  voel  met  inligtingstegnologie,  kan  dit  as 
kragtige  hulpmiddel  ingespan  word  om  hulle  bekend  te  maak  met  die 
"vreemde  taal"  (akademiese  geletterdheid). 

E-leer  maak  ook  fleksieleer  in  terme  van  tyd,  plek  en  selfs  inhoud  (tema,  vlak, 
ens.)  moontlik  (Collis  & Nikolova,  1998).  Die  opvoedkundige  basis  van  die  fleksie- 
leerbeginsel  is  die  konsep  van  geïndividualiseerde  leer  omdat  die  rekenaar  as 
outentieke  leerinstrument  onmiddellike  toepaslike  terugvoer  kan  gee  in  reaksie 
op  'n  spesifieke  individuele  student  se  respons. 


4.  Navorsingsaanpak 

Hierdie  artikel  fokus  op  die  loodsprojek  wat  die  Taalsentrum  in  2006  van  stapel 
gestuur  het  met  die  doel  om  te  bepaal  of  die  voorgestelde  oplossing  vir  die  pro- 
bleem  haalbaar  is  of  nie.  Enkele  navorsingsvrae  is  in  dié  verband  gestel  - vraag  1 
vereis  kwantitatiewe  data  terwyl  vrae  2 tot  8 deur  kwalitatiewe  data  beantwoord 
word. 

4.1  Navorsingsvrae 

Vraag  1:  Sal  studente  wat  vakinhoud  moet  bemeester  deur  e-leer  in  plaas 

van  gewone  kontakonderrig  swakker,  beter  of  dieselfde  daaraan  toe 
wees  as  studente  in  die  kontakonderrigsituasie? 

Vraag  2:  Watter  inhoudelike  en  rekenaartegniese  programaspekte  moet 
aangepas  word  vir  meer  doeltreffende  leer? 

Vraag  3:  Wat  is  die  ideale  kombinasie  van  kontalc-  en  rekenaaronderrig  vir  'n 
volle  geletterdheidskursus? 

Vraag4:  Kan  e-leer  oplossings  bied  vir  die  probleem  van  'n  toenemende 
aantal  taaldienste,  maar  beperkte  personeel  en  kontaktyd? 
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Vraag  5:  Wat  kan  met  e-leer  bereik  word  wat  nie  moontlik  is  met  kontak- 
onderrig  nie? 

Vraag  6:  Spreek  e-leer  inderdaad  tot  die  sogenaamde  digitale  outochtone? 

Vraag  7:  Is  uitbreiding  van  e-leerinhoud  van  akademiese  geletterdheids- 

kursusse  van  een  fakulteit  na  'n  ander  moontlik  ten  einde  voordeel 
uit  die  ekonomie  van  skaal  te  trek? 

Vraag  8:  Kan  e-leer  fleksieleer  bevorder? 

4.2  Navorsingsmetodologie 

Die  navorsingsmetodologie  wat  gevolg  is,  was  'n  eksperiment  binne  die  raam  werk 
van  'n  e-leerloodsprojek  wat  as  basis  gedien  het  vir  die  genoemde  navorsingsfod. 
Die  loodsprojekis  vergestalt  in  'n  akademiese  geletterdheidskursus  (met  die  fokus 
op  leesontwikkeling)  vir  eerstejaarstudente.  Die  kritieke  uitkoms  van  die  kursus 
is  die  ontwikkeling  van  strategiese  leesvermoë  deur  die  toepassing  van  sekere 
hoërorde-taaldenkvaardighede.  Hiervoor  lees  die  studente  talryke  tekste  waarom- 
heen  hulle  probleme  moet  identifiseer  en  oplos,  asook  inligting  versamel,  orden 
en  evalueer.  So  moes  hullebyvoorbeeld 

• die  onbewuste  leesreaksies  (antisipasie,  hipotese,  assosiasie,  verbandleg- 
ging  en  afleiding)  bewustelik  as  denkvaardighede  kan  gebruik  omdat  hulle 
bewus  is  van  die  invloed  van  hul  leserskemata  en  waardesisteem  op  teks- 
interpretasie; 

• 'n  oorhoofse  leesstrategie  ontwikkel  vir  sinvolle  interpretasie  van  tekste  se 
betekenis  op  paragraaf-  en  volteksvlak  deur  uit  die  aantal  leestegnieke  die 
gepaste  een  of  kombinasie  daarvan  te  kies,  soos  bepaal  deur  die  leesdoel; 

• die  kleiner  verband  tussen  temasinne  en  steunsinne  in  een  paragraaf,  asook 
die  oorhoofse 

• verband  tussen  groter  teksdele  met  behulp  van  diskoersleestegnieke  ver- 
staan; 

• onderskeid  tref  tussen  tematies  relevante  en  irrelevante  inligting; 

• die  skiywer  se  organisatoriese  skryfpatrone  (byvoorbeeld  kronologie,  oor- 
eenkoms  en  kontras,  oorsaak  en  gevolg),  asook  die  kommunikasiefunksies 
van  sinne  herken; 

• 'n  teks  effektief  noteer  om  dit  uiteindelik  sinvol  op  te  som. 

4.3  Teikenkursus  en  -groep 

Die  kursus  Verdere  Kommunikasievaardighede  in  Afrikaans  143  (gevorderde  vlak) 
van  dertien  weke  met  vyf  lesings  per  week  is  gevolg  deur  21  eerstejaarstudente  in 
Ingenieurswese.  Dit  is  deur  die  Eenheid  vir  Afrikaans  van  die  Taalsentrum  in  die 
tweede  semester  van  2006  aangebied.  Hierdie  groep  is  geïdentifiseer  vir  die  meer 
gevorderde  taalvaardigheidsvlak  op  grond  van  hul  resultate  in  twee  akademiese 
geletterdheidstoetse,  'n  taalprofielbepaler  en  'n  skryfstuk  in  die  eerste  semester. 
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Tien  studente  was  Afrikaanssprekend,  nege  Engelssprekend  en  twee  sprekers 
van  'n  Afrikataal. 

4.4  Verloop  van  eksperiment 

Die  studente  het  kontakonderrig  ontvang  tot  aan  die  einde  van  week  nege,  toe  20 
studente  'n  voortoets  geskryf  het  wat  die  belangrikste  kursusuitkomste  toets  (een 
van  die  21  studente  het  nie  opgedaag  nie  en  is  dus  vir  kwantitatiewe  navorsings- 
doeleindes  uit  die  eksperiment  gelaat).  Stratifisering  volgens  punte  het  plaas- 
gevind  waarna  die  studente  binne  die  puntestrata  ewekansig  toegewys  is  tot  'n 
eksperimentele  en  'n  kontrolegroep  van  10  studente  onderskeidelik.  In  weke  tien 
en  elf  volg  eersgenoemde  vir  vyf  periodes  per  week  die  e-leersegment  in  'n  toe- 
geruste  rekenaarlaboratorium  waar  die  kurrikuleerder  en  programmeerder  toesig 
hou.  Die  kontrolegroep  gaan  voort  met  die  normale  kontaksessies.  Die  inhoud  is 
vir  albei  groepe  presies  dieselfde:  hersiening  van  die  kursusuitkomste  wat  in 
weke  een  tot  nege  onderrig  is.  Albei  groepe  hou  boek  van  hul  punte  verkry  in 
elke  taak. 

Direk  na  weke  tien  en  elf  skryf  die  studente  'n  natoets  waarin  die  verander- 
likes  soos  vaardigheidstipes,  puntegewig,  tekslengte,  periodegleuwe  en  identi- 
teit  van  toetsafnemers  konstant  gehou  word.  Die  tekstemas  verskil  egter  doelbe- 
wus  om  moontlike  geheue-oordrag  (na  slegs  twee  weke)  uit  te  skakel.  Verder  is 
die  moeilikheidsgraad  van  die  tekste  se  woordeskat  en  sinsbou  doelbewus  ver- 
hoog.  In  week  13  het  die  groepe  weer  verenig  vir  kontakonderrig. 

4.5  Data-insamelmg 

Data  vir  die  kwantitatiewe  navorsingsfokus  is  verkry  deur  die  voor-  en  natoets- 
punte  van  die  eksperimentele  en  kontrolegroepe  met  mekaar  te  vergelyk.  Kwali- 
tatiewe  data  vir  al  die  ander  navorsingsvrae  is  verkry  deur  die  volgende  middele: 

• 'n  leerjoernaal:  die  e-leergroep  het  tydens  elkeen  van  die  5 sessies  in  weke 
10  en  1 1 ná  die  voltooiing  van  elke  taak  daaroor  kommentaar  gelewer  onder 
die  hoofde  Plus,  Minus  en  Interessant,  asook  die  punt  vir  elke  taak 
aangeteken; 

• mondelinge  insette  en  advies  van  die  e-leergroep  tydens  weke  10  en  11, 
veral  oor  programmeringskwessies; 

• waamemingsnotas  deur  kurrikuleerder  en  programmeerder  tydens  hul 
toesig  oor  die  e-leergroep;  en 

• 'n  e-leervraelys,  waarin  die  e-leergroep  hul  ervarings  op  twee  wyses  moes 
uitdruk:  hulle  kon  eerstens  hul  ervarings  van  die  e-leer-  teenoor  die  kon- 
takonderrigsituasie  deur  persentasies  uitdruk  en  skriftelik  motiveer  waarom 
hulle  die  spesifieke  punt  toegeken  het,  en  tweedens  het  hul  kommentaar 
gelewer  op  semi-gestmktureerde  vrae. 
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Tabel  1:  Rekenkundige  gemiddeld  van  toetsuitslae 
uit  ’n  roupuntetotaal  van  50 

Resultate  van  voortoets  Resultate  van  natoets 


Eksperimentele  groep 
Kontrolegroep 


36.1 

36.6 


vertikale  bolkies  loon  95%  verfrouboarheid  intervolle 


e-leergroep 


5.  Navorsingsresultate 

5.1  Kwantitatiewe  resultate 

Die  toetsuitslae  van  die  voor-  en  natoetse  word  in  Tabel  1 uiteengesit.  Die  stu- 
dente  in  die  eksperimentele  groep  (e- 
leer)  sowel  as  die  kontrolegroep  (kon- 
takonderrig)  het  oor  die  algemeen 
swakker  in  die  natoets  gevaar. 

Hierdie  rou  punte  is  aan  statistiese 
analise  onderwerp  met  die  volgende 
resultate  tot  gevolg:  volgens  die  Chi- 
kwadraat-toetse  vir  normaliteit  is  ál  vier 
stelle  data  normaal  versprei  en  voldoen 
dit  ook  aan  die  vereiste  vir  homoske- 
dastisiteit.  Die  data  kon  dus  aan  para- 
metriese  toetse  onderwerp  word.  Ge- 
volglik  is  'n  herhaalde  waarnemings- 
analise  van  variansie  (ANOVA)  ge- 
doen.  Hierdie  resultate  word  in  Figuur 
1 opgesom.  Alhoewel  die  kontrole- 
groep  (kontakonderrig)  se  punte  meer 
gedaal  het,  is  hierdie  effek  volgens  die 
ANOVA  nie  statisties  beduidend  nie 
(p=0.77  vir  die  interaksie-effek).  Die 
een  groep  het  dus  nie  werklik  beter 
presteer  as  die  ander  nie.  Daarom  be- 
hoort  albei  groepe  tot  dieselfde  univer-  26 

sum.  2i) 

Die  afleiding  uit  bostaande  analise  Voortoets 

is  dat  die  e-leergroep  nié  benadeel  is 
deur  e-leerwerk  wat  kontakonderrig 
vervang  nie,  want  hulle  het  dieselfde 
presteer  as  die  kontakgroep.  In  die  lig 
van  die  volgende  konteksspesifieke 
eienskappe  van  die  eksperiment  is 

hierdie  interpretasie  egter  nie  veralgemeenbaar  (oordraagbaar  na  ander  kontek- 
ste)  nie  aangesien 

• slegs  één  dosent  betrokke  was  by  die  onderrig  van  één  klasgroep  - die 
kwaliteit  van  die  dosent  het  ook  'n  invloed  op  die  resultate; 

• die  tydperk  tussen  voor-  en  natoets  (10  uur  in  totaal)  te  kort  was  om  'n 
opvallende  verskil  in  impak  tussen  rekenaar-  en  kontakonderrigmetodiek 
te  identifiseer; 


32.7 

32.3 


S 34 


Natoefs 


Figuur  1 : Gemiddelde  toetspunte  vir  voor- en  natoetse 
apart  vir  die  twee  groepe  studente 
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• die  moontlikheid  van  die  sogenaamde  Hawthorne-effek  ook  in  gedagte 
gehou  moet  word.  Dit  verwys  na  die  situasie  waar  'n  eksperimentele  groep 
opvallend  meer  moeite  doen  om  te  presteer  bloot  omdat  hulle  sielkundig 
gemotiveer  word  deur  die  gefokusde  aandag  wat  hulle  ontvang.  In  die 
onderhawige  eksperiment  sou  mens  ook  die  studente  se  eie  mondelinge 
insette  en  skriftelike  terugvoer  in  die  leerjoernaal  en  die  e-leervaelys  kon 
byvoeg  as  sielkundige  motiveerder. 

Meer  belangrik  egter  vir  die  beplanning  van  die  aanvullende  e-leerprogram  is 
albei  groepe  se  gemiddelde  daling  van  7,6  persentasiepunte  vanaf  voortoets  na 
natoets.  Dit  was  'n  verwagte  tendens  in  die  lig  van  die  doelbewuste  verhoogde 
moeilikheidsvlak  van  die  toets.  Die  doel  van  laasgenoemde  was  om  te  bepaal  of 
die  volle  kursusontwerp  (weke  1-9  plus  hersieningsweke  10  en  11)  die  studente 
voldoende  voorberei  het  op  die  effektiewe  gebruik  van  hulpstrategieë  (soos  kon- 
teks-,  morfologiese  en  sintaktiese  leidrade)  om  die  betekenis  van  abstrakte  akade- 
miese  woordeskat  korrek  af  te  lei.  Die  toetsstatistiekbevestig  die  vermoede  dat 
hierdie  voorbereiding  nie  deeglik  genoeg  was  nie.  Tydens  die  kurrikulering  van 
die  aanvullende  e-leerprogram  sal  hierdie  aspek  van  akademiese  geletterdheid 
meer  aandag  geniet  (verwys  ook  Afdelings  5 en  6). 

5.2  Kwalitatiewe  resultate 

Vir  die  betrokke  navorsers  vorm  hierdie  resultate  die  belangrikste  aspek  van  die 
loodsprojek  omdat  dit  bepaalde  perspektiewe  rondom  navorsingsvrae  2 tot  8 
belig  het.  Veral  het  dit  ook  besluite  bepaal  oor  die  inhoud  en  omvang  van  die 
aanvullende  e-leerprogram  wat  in  2007/8  gekurrikuleer  en  geprogrammeer  word. 
Slegs  die  mees  opvallende  herhalende 

antwoordpatrone  (deur  40%  en  meer  van  die  e-leergroep)  uit  die  twee  skrifte- 
like  bronne  - vraelys  en  leerjoemaal  - word  as  relevant  beskou  en  daarom  ver- 
meld.  Let  veral  op  die  kruisvaliderings  van  response  binne  die  e-leervraelys  self 
en  dan  ook  tussen  vraelys  en  leerjoernaal. 

5.2.1  Die  e-leervraelys 

In  vrae  1, 3, 5 en  6 van  die  vraelys  moes  studente  hul  ervarings  in  persentasies 
uitdruk  en  skriftelik  motiveer  terwyl  hulle  in  Vraag  2 ses  onvolledige  stellings 
skriftelik  moes  voltooi.  Vraag  4 se  resultate  is  nie  gebruik  nie  omdat  die  studente 
die  vraag  op  uiteenlopende  wyses  geïnterpreteer  het  en  die  data  dus  onbetrou- 
baar  is.  Let  wel:  die  onderstaande  bespreking  geskied  om  strategiese  redes  nie  in 
numeriese  vraagvolgorde  nie;  die  sin  daarvan  sal  wel  duidelikblyk. 
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Vraag  1:  Gee  'n  algemene  indruk  van  die  e-leerervaring 
Die  groep  se  algemene  indrukspunt  van  die  leerervaring  is  aan  die  hand  van  'n 
tabel  met  omskryfde  kategorieë  (Uitstekend  = 80-100%;  Goed  = 70-79%;  Bevredi- 
gend  = 60-69%;  Gemiddeld  = 50-59%;  Ondergemiddeld  = Onder  50%)  uitge- 
druken  toon  'nreikwydte  van  70%  tot90%  met'n  totale  gemiddeld  van  77,2%. 

Vraag  5:  In  hierdie  afdeling  word  jy  gevra  om  'n  ideale  onderrigleersituasie 
saam  te  stel  vir  hierdie  module.  Verbeel  jou  jy  het  die  reg  om  self  die 
keuses  te  maak.  Hoe  sou  jou  ideaal  daar  uitsien?  Kies  die  leer- 
ervarings  wat  jy  vir  hierdie  module  sou  insluit.  Skryf  teenoor 
elkeen  die  persentasie  tyd  wat  jy  aan  die  ervaringsou  toeken. 

Die  leerervarings  wat  in  hierdie  keuselys  gebied  word,  is  in  lyn  met  die  instruk- 
sionele  komponente  vir  rekenaargebaseerde  fleksieleer,  soos  gedefinieer  deur 
Collis  en  Nikolova  (1998).  Die  studente  se  positiewe  indrukspunt  in  Vraag  1 
word  in  hierdie  vraag  gevalideer  deur  hul  samestelling  van  'n  ideale  onderrig- 
leersituasie  vir  die  volle  kursus  volgens  hul  keuses  uit  'n  tabel  met  ses  moontlike 
onderrigleersituasies.  Aan  elke  ervaring  van  hul  keuse  moes  hulle  'n  persentasie 
van  die  onderrigleertyd  toeken  sodat  die  totaal  100%  uitmaak.  Figuur  2 stel  die 
drie  ervarings  onder  elkeen  van  die  twee  oorhoofse  kategorieë  - klastyd  en  e-Ieer 
-statisties  voor. 

Die  volgende  twee  voorkeurtendense  is 
besonder  opvallend: 

(i)  Die  leerders  verkies  om  57,7%  tyd  aan  | 
e-leer  te  spandeer  en  42,3%  aan  klastyd.  = 

(ii)  Binne  hierdie  twee  oorhoofse  katego-  1 
rieë  verkies  hu  lle  as  ideale  onderrigleer-  £ 
situasie  'n  kombinasie  van  die  twee  f 
hoogste  voorkeurposisies:  gemiddeld  f 
27,5%  vir  e-leer  en  individuele  werk 
met  dosent  aanwesig  en  21,2%  vir  klas- 
tyd  en  die  lesing. 

Fi 

Uit  bostaande  kan  afgelei  word  dat  hierdie 
spesifieke  groep  studente  81,9%  van  die  totale 
onderrigtyd  in  die  teenwoordigheid  van  die  dosent  wil  deurbring.  Met  die  loods- 
projek  se  kemprobleemstelling  in  gedagte  - toenemende  aantal  taaldienste  maar 
beperkte  kontaktyd  en  personeel  - is  dit  nodig  om  alternatiewe  metodes  van 
kontak  tussen  dosent  en  student  te  ondersoek.  Personeeltekorte  is  oor  die  alge- 
meen  'n  realiteit  op  kampusse  landswyd  en  internasionaal  sodat  slegs  50%  van 
die  totale  onderrigtyd  in  die  teenwoordigheid  van  die  dosent  of  tutor  kan  plaas- 


Klastyd:  Lesing  deur  dosent  (21 .2%) 


Klastyd:  Groepweik  tydens  lesing  (7.9%) 
Klasfyd:  Lindividuele  werk  tydens  lesing  (13.2%) 


e-Leer:  Poarwerk  - dosenl  aanwesig  (12.1%) 


e-leer:  Individuele  werk  - dosent  nanwesig  (27-5%) 


-leer:  Fleksietyd  - geen  dosent  (18.1%) 


iguur  2:  Die  ideale  onderrigleersituasie:  voorkeur- 
tydsbenutting 
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vind  - let  daarop  dat  die  outeurs  nie  hierdeur  die  opinie  huldig  dat  groter  stu- 
dent-dosent-verhoudings  die  enigste  oplossing  vir  die  probleem  is  nie.  Die  oplos- 
sing  word  vanuit  die  perspektief  van  akademiese  bestuurders  juis  gesoek  in  'n 
kombinasie  van  kontak-  en  rekenaaronderrig  wat  in  die  onderste  leerervaring 
van  die  tabel  verteenwoordig  is,  naamlik  e-leerfleksietyd.  As  geen  dosent  teen- 
woordig  is  nie,  kan  die  student  dus  enige  tyd  en  op  enige  plek  met  die  aanvul- 
lende  oefeninge  werk.  Daar  is  twee  voordele  hieraan  verbonde:  in  die  eerste  plek 
bring  dit  verligting  vir  die  dosent;  tweedens  bied  dit  die  student  die  geleentheid 
om  op  sy/haar  eie  tyd,  plek  en  tempo  te  leer  (Collis  & Nikolova,  1998).  Daarom  is 
dit  verblydend  dat  die  studente  wel  aan  hierdie  leerervaring  die  tweede  voorkeur- 
posisie  gegee  het  binne  die  e-leerkategorie.  Die  studente  se  duidelike  voorkeur 
vir  individuele  bo  groepwerk  (geldig  vir  beide  kontak-  en  e-leersituasies)  behoort 
ook  verreken  te  word  in  herkurrikuleringsaktiwiteite.  Sodanige  ondersoek  val 
egter  buite  die  begrensing  van  hierdie  artikel. 

Vraag  6:  Gee  redes  waarom  jy  die  besondere  keuse  van  leerervarings  gemaak 
hetin  vraag5. 

Die  skriftelike  redes  vir  hul  voorkeurtydsbenutting  valideer  die  bogenoemde 
tweede  voorkeurposisie.  Vergelyk  die  volgende  opvallende  tendense  wat  kort- 
liks  beskryf  sal  word. 

Redes  vir  e-leer : individuele  werk  met  dosent  aanwesig 

(i)  Met  individuele  werk  op  die  rekenaar 

- voel  hulle  nie  gedruk  om  dadelik  'n  dosent  se  vraag  te  beantwoord 
nie; 

- bring  dit  die  nodige  inoefening  vir  bemeestering  van  vaardighede; 

- is  die  rekenaarterugvoer  "net  so  goed  soos  die  dosent  s'n"; 

- kan  hulle  werk  teen  hul  eie  individuele  spoed. 

(ii)  Die  dosent  as  fasiliteerder  is  nodig  om  sekere  konsepte  te  verduidelik.  Let 
hier  op  'n  moontlike  ambivalensie:  hoewel  81,9%  van  die  studente  aange- 
dui  het  dat  hulle  die  onderrigtyd  in  die  teenwoordigheid  van  die  dosent 
wil  deurbring,  het  50%  van  hierdie  selfde  groep  aangedui  dat  hierdie 
voorkeur  egter  arbitrêr  is,  omdat  hulle  ook  later  sekere  vrae  aan  hom/haar 
sou  kon  stel.  Dit  hou  vermoedelik  verband  met  die  studente  se  positiewe 
indruk  van  die  kwaliteit  van  die  rekenaarterugvoer  as  "net  so  goed  soos 
die  dosent  s'n"  en  daarom  dus  die  fleksietyd  in  tweede  voorkeurposisie. 

Redes  vir  klastyd:  lesing  deur  dosent 

'n  Eerste  verduideliking  deur  die  dosent  is  noodsaaklik  vir  begrip  van  konsepte 
of  vaardighede  voor  die  individuele  inoefening  op  die  rekenaar. 
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Vraag  3:  Gedurende  die  meeste  e-leerinteraksies  is  teksterugvoer  na  korrekte 
en  verkeerde  respons  gegee.  In  watter  mate  het  jy  jou  toegespits  op 
die  terugvoer? 

• Na  'n  korrekte  respons: 

• Na  'n  verkeerde  respons  op  'n  vraag  met  net  een  kans  om  te  antwoord: 

• Na  'n  tweede  verkeerde  respons  op  'n  vraag  met  twee  kanse  om  te  ant- 
woord: 

Studente  moes  aan  die  hand  van  'n  tabel  met  persentasie-kategorieë  aantoon  in 
watter  mate  hulle  die  rekenaarterugvoer  aandagtiggelees,  geïnterpreteer  en  toege- 
pas  het  in  daaropvolgende  antwoord-strategieë.  Na  'n  korrekte  respons  het  stu- 
dente  gemiddeld  54%  aandaggegee.  Na  'n  verkeerde  respons  op  'n  vraagmet  net 
een  kans  om  te  antwoord  het  die  toegespitsde  aandag  dramaties  verhoog  na  84,4% . 
Dit  het  selfs  nóg  verder  verhoogna  87,6%  as  hulle  'n  tweede  verkeerde  respons 
gegee  het  op  'n  vraag  waar  hulle  twee  kanse  gehad  het  om  te  antwoord.  Let 
daarop  dat  die  terugvoer  in  'n  vraag  met  twee  kanse  om  te  antwoord  altyd  'n 
ingeboude  leidraad  het  wat  elke  student  lei  tot  nadenke  oor  sy/haar  fout,  asook  'n 
herlees  van  die  betrokke  teksdeel  om  die  tweede  kans  meer  effektief  te  benut. 

Die  opvoedkundige  voordeel  wat  hierdie  soort  geïndividualiseerde 
rekenaarterugvoer  het  bo  konvensionele  kontakonderrig  spreek  dus  duidelik  uit 
die  verhoogde  aandagpatroon  wat  uit  die  statistiek  blyk.  In  Afdeling  6 sal  die 
beginsel  meer  omvattend  bespreek  word. 

Vraag  2:  Voltooi  die  volgende  sinne  aangaande  die  e-leerervaring 

• Ekhetgehou  van ... 

• Ekhetnie gehou  van  ... 

• Ek het  dit maklik gevind  dat ... 

• Ekhetditmoeilikgevinddat ... 

• Wat  ek  sou  verander,  is  . . . 

• Die  vlakvantaalgebruikwas  ... 

Slegs  drie  kategorieë  binne  hierdie  vraag  het  herhalende  antwoordpatrone  opge- 
lewer  en  word  dus  as  relevant  beskou  vir  die  kwalitatiewe  navorsingsfokus.  Die 
verskynsel  van  kruisvalidering  van  data  in  ander  vrae  is  in  ál  drie  opvallend  en 
word  derhalwe  uitgelig: 

(i)  Ek  het  gehou  van  . . . 

- die  onmiddellike  geïndividualiseerde  rekenaarterugvoer;  het  daar- 
deur  "makliker  en  meer  geleer"  en"foute  dadelikverstaan"  (70%); 

- die  stilte  waarin  (ek)  "beter  kan  konsentreer"  en  dus"  beter  leer" 
(40%). 
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Uit  bogenoemde  is  dit  duidelik  dat  die  eerste  voorkeur  die  statistiek  oor  die 
toename  in  studente  se  aandagtige  lees  van  terugvoer  (Vraag  3)  kruisvalideer. 
Albei  voorkeure  kruisvalideer  ook  hulle  57,7%-tydsvoorkeur  vir  e-leerwerk  bo 
die42,3%  klastydvoorkeur  (Vraag5). 

(ii)  Wat  ek  moeilik  gevind  het,  was  . . . 

- akademiese  woordeskat  (die  5 tweedetaalsprekers) 

- Resvanstudentevulniksinnie. 

(iii)  Vlak  van  taalgebruik,  was  . . . 

- soms  moeilik  maar  uitdagend  danksy  woordeboeke  (dieselfde  5 
tweedetaalsprekers) 

- regverdig  (4  moedertaalsprekers) 

- Een  student  vul  niks  in  nie. 

Die  kruisvalidering  tussen  hierdie  twee  kategorieë  is  opvallend  en  is  bevestig  met 
verdere  validering  wat  geskied  het  aan  die  hand  van  die  tweedetaalsprekers  se 
deurlopende  kommentaar  in  hul  leerjoemale,  naamlik  dat  hulle  met  die  woorde- 
skat  probleme  ondervind  terwyl  die  moedertaalsprekers  niks  daarvan  gemeld  het 
nie.  Hierdie  woordeskatprobleem  - veral  by  abstrakte  meer  akademiese  woorde  - 
is  'n  bepalende  faktor  in  die  algemene  daling  van  7,6  persentasiepunte  vanaf  voor- 
toets  na  natoets.  Let  egter  daarop  dat  alle  studente  se  punte  deel  vorm  van  hierdie 
daling,  met  ander  woorde  ook  die  moedertaalsprekers  s'n.  Die  afleiding  lê  voor 
die  hand:  woordbegrip  is  nie  net  die  resultaat  van  blote  dekodering  van  woorde 
as  'n  spesifieke  taal  se  klank-  en  skriftekens  nie.  Veel  eerder  hang  dit  saam  met 
hoërorde-leesstrategieë  wat  die  effektiewe  gebruik  van  diskoers-,  konteks-,  mor- 
fologiese  en  sintaktiese  merkers  insluit.  Dit  is  'n  oorhoofse  leesvaardigheid  waarmee 
moedertaalstudente  net  soveel  probleme  ondervind.  Die  lae  puntegemiddeld  van 
die  eksperimentele,  asook  die  kontrolegroep  in  sekere  hoërorde-leestake  in  die 
vyf  oefensessies  bevestig  hierdie  bevinding  verder  (sien  5.2.2.2). 

5.2.2  Die  leerjoernaal 

Studente  het  moeite  gedoen  om  by  elke  taak  onder  die  drie  hoofde  vrywillige 
terugvoer  te  gee  wat  dikwels  aangevul  is  deur  spontane  mondelinge  insette. 

5.2.2. 1 Programmeringsaanpassings 

Hierdie  aspek  van  die  navorsing  kan  nie  sinvol  in  die  artikel  bespreek  word 
sonder  die  leser  se  insae  in  die  spesifieke  taaktipe  en  programmeringsformaat  ter 
sprake  nie.  Slegs  twee  voorstelle  deur  die  studente  word  egter  uitgelig,  omdat  dit 
goeie  voorbeelde  is  van  hoe  die  rekenaar  as  leerinstrument  die  swak  leser  ook 
deur  die  programmering  kan  bemagtig.  Die  volgende  twee  behoeftes  spruit  juis 
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uit  spanning  wat  studente  ervaar  het  weens  die  baie  leeswerk  van  opdragte, 
tekste  en  terugvoer  wat  vir  studente  met  leesprobleme  baie  ontsenuend  kan  wees: 

(i)  Selfs  al  is  die  student  al  besig  met  die  uitvoer  van  die  opdrag,  moet  die 
oorspronklike  opdragformulering  steeds  (albei  in  verkleinde  formaat)  sigbaar 
wees  of  met  die  klik  van  'n  knoppies  kan  verskyn  en  verd  wyn. 

(ii)  By  veelkeusestellings  van  komplekse  aard  moet  die  oorspronklike  stelling 
en  opsies  steeds  sigbaar  wees  as  die  student  terugvoer  ontvang  na  sy  respons.  Die 
rede:  die  terugvoer  verwys  dikwels  na  die  inhoud  van  die  stelling  en/of  gee  'n 
subtiele  leidraad  as  die  student  'n  tweede  kans  het  om  te  antwoord.  In  die  lei- 
draad  word  opsies  soms  teen  mekaar  afgespeel. 

5. 2.2.2  Inhoudelike  aanpassings  by  sekere  vaardigheidstipes 

Die  eksperimentele  en  kontrolegroepe  se  punte  vir  elke  taak  in  elkeen  van  die  vyf 
sessies  is  saamgevoeg,  en  die  gemiddeld  daarvan  is  uitgewerk.  Daarvolgens  is  die 
vaardigheidstipes  in  'n  rangorde  geplaas  van  laagna  hoog.  Hierdie  data  vorm  die 
vertrekpunt  vir  die  ontwerp  en  programmeringin  2007/8  van  die  aanvullende  e- 
leerprogram.  Veral  die  vaardigheidstipes  in  die  lae  sestigs  en  vyftigs  (sien  Tabel  2) 
sal  met  'n  verskeidenheid  mikro-oefeninge  bygewerk  word.  'n  Asterisk  beteken 
dat  die  e-leergroep  in  daardie  oefening  'n  tweede  kans  gehad  het  om  te  antwoord 
en  dan  heelwatbeter  gevaar  het  as  die  klasgroep.  Die  gemiddelde  punt  van  daar- 
die  vaardigheidsoefening  sou  dus  nog  laer  gewees  het  as  hulle  - soos  die  klas- 
groep  - net  een  antwoordkans  gehad  het. 


Tabel  2:  Problematiese  vaardigheidstipes 


Krities  lees  om  geldige  en  ongeldige  afleidings  van  mekaar  te  onderskei 

50,5% 

Diskoerslees  om  tekspatrone  en  -organisasie  te  identifiseer 

52,5% 

Analities  lees  om  die  tematiese  funksie  van  woorde  vas  te  stel 

52% 

Diskoerslees  om  uit  opsiesinne  met  komplekse  sinsbou  en  verskuilde  herhaling 

die  beste  temasin  te  kies 

53% 

Diskoerslees  om  verbande  tussen  tematiese  sleutelwoorde  te  identifiseer 

53,4% 

Analities  en  diskoerslees  om  in  sluitingsoefeninge  abstrakte  (akademiese)  woorde 

korrek  te  identifiseer 

53,4% 

Soeklees  om  sekere  inligting  binne  'n  tydsgrens  te  identifiseer  (spoedlees) 

56% 

* Analities  en  diskoerslees  om  sinspare  met  korrekte  skakelwoorde  te  verbind 

58,5% 

* Krities  lees  om  die  kommunikasiefunksie  van  stellings  oor  die  teks  te  herken 

60,4% 

* Diskoerslees  om  groter  tekseenhede  binne  ’n  kopkaart  te  identifiseer 

60,5% 
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Die  leser  word  herinner  aan  die  studente  se  probleme  met  veral  abstrakte, 
sogenaamde  "hoë  woorde",  wat  die  tweedetaalsprekers  uitgewys  het  in  Vraag  2 
van  die  Vraelys,  maar  wat  by  al  die  studente  in  albei  groepe  geïdentifiseer  is  na 
aanleiding  van  die  gemiddelde  7,6  persentasiepuntdaling  van  voor-  na  natoets. 
Onderliggend  aan  die  oneffektiewe  hantering  van  bogenoemde  vaardigheidstipes 
is  dan  ook  die  opvallende  gebrek  aan  abstrakte  woordeskat  kenmerkend  van  die 
akademiese  konteks. 


6.  Terugblik  op  navorsingsvrae 

Die  navorsingsresultate  van  hierdie  studie  is  duidelike  rigtingwysers  na  die  krea- 
tiewe  oplossing  vir  die  Universiteit  Stellenbosch  se  probleem  van  toenemende 
fakultêre  taaldienste,  maar  beperkte  kontaktyd  en  personeel.  Terselfdertyd  wys 
die  resultate  ook  die  rigting  aan  vir  die  manier  waarop  die  Taalsentrum  die 
voordele  van  e-leer  kan  benut  om  'n  aanvullende  kursusverwante  e-leerprogram 
te  ontwikkel.  Die  kwantitatiewe  voor-  en  natoetsresultate  het  statisties  uitgewys 
dat  die  loodsprojek  se  e-leerkomponent  van  vyf  sessies  nie  die  e-leergebruikers 
enigsins  benadeel  het  nie.  Aangesien  dit  kontaksessies  volledig  vervang  het,  spreek 
dit  dan  vanself  dat  'n  e-leerprogram  wat  konsepte  of  vaardighede  aanvullend 
inoefen  eers  nadat  dit  ingelei  en  toegepas  is  in  die  gewone  lesingsituasie  die 
studente  nog  minder  sal  benadeel. 

Dit  is  egter  veral  by  die  kwalitatiewe  data  waar  dit  duidelik  word  dat  die  e- 
leermedium  sekere  kreatiewe  oplossings  kan  bring  om  leer  te  bevorder.  Vervol- 
gens  word  die  verskillende  navorsingsvrae  of  aspekte  daarvan  (in  skuinsdruk 
aangedui)  as  'n  integrale  deel  van  hierdie  psigodinamika  bespreek. 

Eerstens  spreek  e-leer  inderdaad  tot  die  nuwe  digitale  outochtoon.  Dit  is  duidelik 
uit  die  studente  se  voorkeur  vir  e-leerwerk  (57,7%)  bo  klastyd  (43,3%)  as  ideale 
onderrigleerkombinasie  vir  'n  volle geletterdheidskursus.  Hul  algemene  voorkeur  vir 
die  onmiddellike  geïndividualiseerde  rekenaarterugvoer  wat  hulle  "makliker  en 
meer  " laat  leer,  hul  "foute  dadelik"  laat  "verstaan"  en  boonop  "net  so  goed  soos 
die  dosent  s'n"  is,  lig  die  sterk  opvoedkundige  onderrigrol  van  die  rekenaar  as 
outentieke  leerinstrument  duidelik  uit.  Vermoedelik  is  die  groep  se  positiewe 
indrukspunt  van  77,2%  vir  die  vyf  sessies  juis  te  danke  aan  die  fasiliteringsrol  wat 
die  rekenaar  in  hierdie  sessies  gespeel  het. 

Geen  aanvullende  e-leerprogram  sal  sonder  hierdie  fasiliteringsrol  slaag  nie, 
want  dit  impliseer  'n  baie  spesifieke  betekenis  van  die  konsep  evaluering  - in 
Latyn  die  uitbring  ( ex ) van  die  student  se  krag  ( valere ).  Hierdie  betekenis  word 
geïllustreer  in  die  volgende  reaksies  van  studente  op  die  geïndividualiseerde  en 
onmiddellike  terugvoer  wat  die  e-medium  bied  -let  op  die  studente  se  toename 
in  toegespitsde  aandag,  tussen  hakies  aangedui: 
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(i)  As  die  student  'n  item  korrekbeantwoord,  reageer  die  rekenaar  met  'n 
positiewe  woord  (byvoorbeeld  Kolskoot!)  en  verskaf  die  rede  vir  die 
korrekte  antwoord.  [54%] 

(ii)  In  die  onmiddellike  terugvoer  ná  'n  eerste  foutiewe  respons  wys  dit  die 
korrekte  rigting  aan  metbehulp  van  leidraadkommentaar  waaroor  die 
student  dan  nadink  voordat  hy/sy  'n  tweede  keer  probeer.  [84,4%] 

(iii)  As  die  student  dan  korrek  reageer,  reageer  die  rekenaar  soos  in  (i)  hierbo. 

(iv)  Na  'n  tweede  foutiewe  respons  verskaf  die  rekenaar  die  rede  daarvoor 
plus  die  korrekte  respons  én  die  rede  daarvoor.  [87,6%] 

Die  statistiese  toename  in  aandag  illustreer  ten  beste  die  krag  van  die  rekenaar  as 
outentieke  leerinstrument.  Selfs  as  die  student  die  totale  punt  vir  die  oefening 
sien,  lig  die  rekenaar  die  kriteria  of  aspekte  uit  waaraan  die  student  nog  aandag 
moet  skenk.  Die  rekenaar  onderrig  dus  steeds  terwyl  dit  evalueer,  en  lei  sodoende 
die  student  deur  'n  proses  van  nadenke  en  ontwikkelende  insig.  Dit  is  ware 
bemagtiging  - inderdaad  ver  verwyder  van  die  rekenaar  as  toetsmasjien  wat 
bloot  'n  akkumulatiewe  klaspunt  genereer  sonder  dat  studente  insig  in  hul  ge- 
breke  én  hul  krag  kry.  e-Leer  bereik  inderdaad  groter  toegespitste  aandag  van  baie 
studente gelyktydig as  gevolg  van  dié  intense geïndividualiseerde  terugvoer  wat  prakties 
onuitvoerbaar  is  in  kontakonderrig  met  groot  klasse. 

Die  impak  van  hierdie  evalueringskonsep  van  e-leer  op  die  ontwerp  van  'n 
kursus  is  enorm  in  terme  van  kwaliteitsbeplanning  en  die  arbeidsintensiwiteit 
daarvan.  Dit  vereis  van  die  ontwerper  om  deurentyd  vanuit  die  leerder  se  be- 
hoeftes  na  die  lesontwerp  te  kyk  om  vas  te  stel  watter  inhoudelike  en  rekenaarteg- 
niese  programaspekte  aangepas  moet  word  vir  tneer  doeltreffende  inoefening  van  lees- 
vaardighede.  Die  volgende  vrae  dwing  die  ontwerper  om  krities  te  besin: 

• Met  watter  foutiewe  konsepte/vaardighede  worstel  studente? 

• Waar  inmy  ontwerp  is  daar  gapings  omdat  eknie  die  presiese  omvangvan 
die  studente  se  taalprobleem  besef  het  nie? 

Dit  is  hier  waar  die  opvallende  kruisvalidering  tussen  kwalitatiewe  data  in  die 
vraelys  en  leerjoernaal  en  kwantitatiewe  data  oor  lae  natoetspunt  en  punte  in 
sekere  vaardigheidstipes  van  onskatbare  waarde  was  (4.2.2.2).  Dit  het  duidelik 
uitgewys  dat  die  ontwerper  vir  die  aanvullende  e-leerprogram  meer  oefeninge 
moet  bedink  rondom  oorhoofse  leesstrategieë  wat  studente  se  gebrekkige  ab- 
strakte  woordeskat  en  onvermoë  om  betekenis  in  konteks  af  te  lei  - veral  in  tekste 
met  komplekse  sinsbou  - aanspreek. 

Met  die  opvoedkundige  voordeel  van  intense  geïndividualiseerde  onderrig 
aan  baie  studente  gelyktydig  via  'n  aanvullende  e-leerprogram  - soos  dit  in  die 
voorafgaande  bespreking  duidelik  geword  het  - is  die  kernprobleem  van  beperkte 
kontaktyd  vir  akademiese geletterdheidskursusse  infakulteite  in  beginsel  dus  opgelos. 
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Die  sleutelwoord  is  "geïndividualiseerd",  want  dit  beteken  dat  elke  studerende 
individu  - sonder  die  teenwoordigheid  van  die  dosent  in  eie  tyd  en  op  enige 
plek  in  een  van  die  vele  intemettoegangsgebiede  op  kampus,  in  koshuiskamer  of 
in  'n  rekenaarlaboratorium  kan  sit  en  werkmet  toegespitsde  aandag. 

Daar  is  egter  twee  ononderhandelbare  voorwaardes  vir  die  sukses  van  hierdie 
fleksieleerbeginsel: 

(i)  die  e-leerprogram  moet  beantwoord  aan  die  opvoedkundige 
evalueringsbeginsel  (hierbo  bespreek)  wat  die  rekenaar  as  fasiliteerder 
van  die  outentieke  leerproses  voorveronderstel;  en 

(ii)  die  inhoud  moet  tematies  geïntegreer  wees  met  die  beperkte  aantal 
kontaklesings  wat  die  kursus  se  vaardighede  en  konsepte  inlei. 

Die  leser  word  ook  herinner  aan  die  feit  dat  die  studente  in  hul  terugvoer  oor 
voorkeurtydsbenutting  vir  die  ideale  onderrigleerkombinasie  (Vraag  5 in  vrae- 
lys)  tweede  voorkeur  gegee  het  aan  e-leer  en  fleksietyd  en  in  hul  geskrewe  motive- 
rings  opvallend  ambivalent  was  oor  die  dosent  se  teenwoordigheid  al  dan  nie 
(5.2.1).  Daar  sou  dus  veralgemeen  kon  word  dat  die  gemiddelde  student  kam- 
puswyd  nie  beswaar  sal  maak  teen  fleksieleer  op  die  rekenaar  as  daar  aan  boge- 
noemde  twee  voorwaardes  voldoen  word  nie. 

Die  belangrikste  kriterium  van  geslaagde  fleksieleer  lê  opgesluit  in  die  kon- 
sep  van  gedifferensieerde  leer  wat  die  student  lei  tot  verantwoordelike  nadenke 
- die  basis  van  geslaagde  selfstandige  studie  (Collis  & Nikolova,  1998).  Die  aanvul- 
lende  e-leerprogram  se  bydrae  tot  fleksieleer  lê  dus  veral  op  die  vlak  van  gedifferensieerde 
vaardigheidsvlakke.  Die  proses  verloop  soos  volg:  'n  aantai  sessies  word  ten  opsigte 
van  inhoud  en  vaardighede  gekoppel  aan  sekere  lesinginhoude.  Studente  besluit 
self  hoe  goed  hulle  die  bekendstelling  van  vaardighede  en  konsepte  in  die  lesing 
verstaan  het.  Daarvolgens  kies  hulle  op  watter  vlak  hulle  elke  sessie  aanpak.  Die 
sessies  begin  met  'n  voortoets  en  eindigmet  dieselfde  inhoud  as  natoets.  Tussen 
hierdie  twee  produktoetse  lê  die  reeks  oefeninge  waarin  studente  deur  geïndivi- 
dualiseerde  terugvoer  geëvalueer  en  onderrig  word.  Die  program  sorg  dus  vir  'n 
unieke  ketting  van  toetsing  (leer  as  produk),  evaluering  (leer  as  proses)  en  weer 
toetsing. 

Die  voortoetspunt  dui  aan  of  studente  té  hoog  of  té  laag  mik  en  dus  liewers  die 
laer  of  die  hoër  vlak  moet  doen.  Die  natoetspunt  dui  aan  of  studente  die  reeks 
oefeninge  moet  herhaal  en/of  die  laer  of  hoër  vlak  moet  probeer.  Alles  bly  hulle 
eie  keuse,  want  om  suksesvol  te  leer  is  hulle  verantwoordelikheid.  Niemand 
dwing  hulle  om  al  die  oefeninge  te  doen  of  dit  in  'n  spesifieke  volgorde  te  doen 
nie,  behalwe  as  dit  vooraf  so  aangedui  word  weens  die  oorsaaklike  verband  tus 
sen  sekere  vaardighede, 

Hierdie  gedifferensieerde  benadering  wil  veral  dié  studente  wat  sukkel  met 
akademiese  woordeskat  en  komplekse  sinsbou  bernagtig.  In  spoedleesoefeninge 
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kry  hulle  langer  tyd;  tekste  is  korter;  en  moeilike  woorde  word  deur  woordver- 
klarings  of  in  die  terugvoer  verhelder.  Vlak  2-studente  se  spoedleesoefeninge 
geskied  uiteraard  in  'n  korter  tydsbestek,  tekste  is  langer  en  hulle  kry  minder 
woordverklarings.  Alle  studente  het  ook  toegang  tot  'n  woordgids  wat  veral 
moeiliker  abstrakte  woorde  en  belangrike  sleutelkonsepte  binne  die  kur- 
susraamwerk  bondig  verduidelik. 

Weens  die  arbeidsintensiewe  ontwerp  en  programmering  van  hierdie  soort 
fleksieleerprogram  met  sy  geïndividualiseerde  terugvoer  en  ingeboude  steun- 
meganismes  moet  die  moontlikheid  oorweeg  word  om  die  volle  geïntegreerde  akade- 
miese  geletterdheidskursus  ook  na  ander  fakulteite  uit  te  brei.  Die  ekonomie  van  skaal 
(groot  aantal  studente)  sal  die  kostes  verbonde  aan  ontwerp  en  programmering 
regverdig.  Die  vraag  is:  Is  dit  prakties  uitvoerbaarl 


7.  Slotopmerkings  en  gevolgtrekking 

Die  kursus,  wat  reeds  in  2006  in  die  Ingenieursfakulteit  geïmplementeer  is  as 
semesterkursus,  word  sedert  2007  ook  in  die  Fakulteit  Natuurwetenskappe  aange- 
bied  as  'n  jaarkursus  met  twee  kontaklesings  per  week.  Uitbreidings  na  alle 
fakulteite  is  wel  prakties  uitvoerbaar  as  gevolg  van  die  generiese  aard  van  die 
kritieke  uitkomste  en  die  spesifieke  kursusuitkomste  van  'n  akademiese  geletterd- 
heidskursus.  Die  beginsel  van  ekonomie  van  skaal  kan  dus  toegepas  word,  ten 
einde  finansiële  voordeel  te  trek.  Hierdie  beginsel  dryf  juis  die  onmiddellike 
uitvloeisels  van  die  projek,  naamlik  dat  die  ontwerper  en  programmeerder  met 
die  kwalitatiewe  data  uit  die  navorsingsprojek  as  stimulus  die  aanvullende  e- 
leerprogram  sodanig  aanpas  dat  dit  na  meer  fakulteite  en  moontlik  ook  ander 
instellings  uitgebrei  kan  word,  ongeag  die  inhoud  van  die  vakdissipline. 

Wat  egter  nie  uit  die  oog  verloor  kan  word  nie,  is  die  waarskynlik  negatiewe 
impak  van  'n  sogenaamde  koste-effektiewe  benadering  (minder  dosente  en  groter 
klasse)  op  die  kwaliteit  van  onderrig  en  leer,  maar  soos  reeds  genoem  kon  hierdie 
aspek  nie  binne  die  beperkinge  van  hierdie  studie  ondersoek  word  nie. 

Wanneer  die  rekenaar  werklik  as  fasiliteerder  van  die  student  se  leerproses 
ingespan  word,  word  die  student  - as  digitale  outochtoon  - in  'n  digitale  om- 
gewing  ontmoet  waarmee  hy/sy  bekend  is.  Flierdie  bekende  omgewing  behoort 
daartoe  by  te  dra  dat  die  student  se  ervaring  van  akademiese  taal  as  bevreem- 
dende  diskoers  omgeskakel  kan  word  na  akademiese  taal  as  toegangsportaal. 


Bibliografie 

Bencze,  J.L.  2005.  Constructivism,  [Intyds].  Beskikbaar  by:  http://leo.oise.utoronto.ca/~lbencze/ 
Constructivism.html.  Besoek  op  9 Desember  2005. 

Blacquiére,  A.  1989.  Reading  for  survival:  text  and  the  second  language  student.  South  African 
Joumal  forHigher  Education  3(l):73-82. 


172 


Ensovoort:  ]aargang  1 1 , nommer  2,  2007 


Blanckenberg,  H.C.  1999.  Die  onderrig  van  Afrikaans  as  altematiewe  taal  in  'n  uitkomsgebaseerde 
onderwysbenadering.  Ongepubliseerde  VDO-Studiegids.  Johannesburg:  Nasionale  Private 
Kolieges. 

Botha,  H.L.  & Cilliers,  C.D.  1999.  Preparedness  for  university  study:  Designing  a thinking  skills 
test.  SouthAfrican  JournalofHigherEducation  13(1):144 — 152. 

Butler,  H.G.  & Van  Dyk,  T.J.  2004.  An  academic  English  language  intervention  for  firstyear 
engineering  students.  South  African  Joumal  ofLinguistics  22(1  &2):l-8. 

Collis,  B.  & Nikolova,  1. 1998.  Flexible  leaming  and  design  of  instruction.  British  Joumal  of 
Educational  Technology  29(1  ):59— 72. 

Leibowitz,  B.  2001 . Students'  prior  leaming  and  their  acquisition  of  academic  literacy  at  a 
multilingual  South  African  university.  Ongepubliseerde  doktorale  proefskrif.  Sheffield: 
University  of  Sheffield. 

Macdonald,  C.A.  1990.  Crossing  the  threshold  into  standard  three  in  black  education:  The  consolidated 
main  report  oftiie  threshold  project.  Pretoria:  Human  Sciences  Research  Council. 

Mamaila,  K.  2001 . Rl,3bn  spent  on  dropout  students.  Star,  15  May  2001 :1. 

McKenzie,  K.  & Schweitzer,  R.  2001 . Who  succeeds  at  university?  Factors  predicting  academic 
performance  in  first  year  Australian  imiversity  students.  Higher  Education  Research  and 
Deoelopment  20(1):21— 33. 

Natal  Witness.  2004.  Illiterate  matrics.  Natal  Witness,  27  September:8. 

Perkins,  D.M.  1991 . Improvement  of  reading  and  vocabulary  skills  at  the  University  of  Transkei. 
South  African  Jourruú  ofEducation  11  (4):231— 235. 

Pierce,  W 2003.  Student  preparednessforcollege  work  [Intyds].  Beskikbaar  by:  http://academic.pg. 
cc.md.us/~wpeirceMCCCTR/ctrep~l.html.  Besoekop  24  Maart  2004. 

Prensky,  M.  2001.  Digital  natives,  Digital  immigrants.  On  the  Horizon.  9(5):1-15. 

Pretorius,  E.J.  1995.  Reading  as  an  interactive  process:  Implications  for  studying  through  the 
medium  of  a second  language.  Communicatio  21(2):33-43. 

Rademeyer,  A.  2005a.  Te  veel  studente  "hoort  nie  werklik  op  universiteit":  Tot  soveel  as  30%  van 
eerstejaars  het  nie  vrystelling  nie.  Die  Burger,  14  Mei:4. 

Rademeyer,  A.  2005b.  Klas  van  2008.  Insig,  28  Februarie:40. 

Vorster,  J.  & Reagan,  T.  1990.  On  the  lexical  development  of  L1  and  L2  speakers.  SouthAfrican 
Joumal  ofLinguistics  9(3):80 — 84. 

Zamel,  V.  & Spack,  R.  (eds.)  1998.  Negotiating  academic  literacies:  Teaching  and  learning  across 
languages  and  cultures.  New  Jersey:  Lawrence  Erlbaum. 


Voorskrifte  aan  skrywers 


Ensovoort  ontvang  graag  artikels,  boekbespre- 
kings  en  skeppende  skryfwerk  (gedigte,  prosa 
en  kort  dramas)  met  die  oog  op  publikasie.  Die 
fokus  val  op  die  Afrikaanse  en  Nederlandse  lite- 
ratuur  binne  die  konteks  van  sowel  die  breë 
Suid-Afrikaanse  as  die  Nederlandse  samele- 
wings.  Bydraes  van  sosiokulturele,  omge- 
wingsletterkundige,  toegepaste  taalkundige  of 
vergelykende  aard  is  ook  ter  sake. 

Ofskoon  bydraes  in  Afrikaans,  Nederlands 
en  Engels  voorkeur  kry,  word  bydraes  in  die 
ander  tale  van  Suid-Afrika  ook  verwelkom. 

Ensovoort  is  geakkrediteer  vir  subsidie  aan 
universiteite  en  navorsingsuitsette.  Bladgeld 
sal  verhaal  word  van  die  instansies  van  skry- 
wers  van  wie  bydraes  verskyn  wat  vir  subsi- 
die  kwalifiseer. 

Bydraes  van  hoogstens  6 000  woorde  moet 
elektronies  aan  die  redaksie  gestuur  word  en 
in  een-en-’n-half  spasiëring  met  Times  New 
Roman  12-punt  lettertipe  met  inspringende 
paragrawe  en  sonder  ’n  oop  reël  tussen  para- 
grawe  getik  wees.  Artikels  moet  ook  'n  verta- 
ling  van  die  titel  en  'n  opsomming  van  150  tot 
200  woorde  in  Engels  bevat. 

Bydraes  word  deur  minstens  twee  keurders 
geëvalueer.  Hoewel  die  keurders  se  verslae  aan 
die  bydraers  gestuur  kan  word,  sal  die  keur- 
ders  se  identiteit  deurgaans  vertrouhk  gehou 
word. 

Alle  bydraes  moet  reeds  tydens  die  voor- 
legging  gereed  wees  vir  publikasie  en  behoor- 
lik  taalkundig  versorg  wees.  Hoë  eise  word  ten 
opsigte  van  gehalte  en  taalkundige  afgerond- 
heid  gestel.  Die  redaksie  behou  hom  wel  die 
reg  voor  om  bydraes  (indien  nodig)  na  evalu- 
ering  taalkundig  te  versorg,  maar  eerbiedig  die 
individualiteit  van  outeurs  sover  moontlik. 

Die  Harvard-metode  moet  vir  verwysings 
gebruik  word.  Bronverwysings  in  die  teks 
moet  dus  die  van(ne)  van  die  skrywers(s),  jaar 


van  publikasie  en  bladsynommers  tussen  ha- 
kies  bevat,  byvoorbeeld  (Cloete,  1963:6). 

Die  nommers  van  eindnotas  verskyn  as 
boskrifte  sonder  hakies  en  regs  van  enige 
leestekens  in  die  teks.  Die  eindnotas  verskyn 
aan  die  einde  van  die  teks  ná  die  bibliografie. 

Volledige  besonderhede  van  die  literatuur 
waarna  in  die  teks  verwys  word,  moet  aan  die 
einde  van  die  artikel  onder  die  opskrif  "Bibli- 
ografie"  geplaas  word.  E>ie  literatuur  word  al- 
fabeties  volgens  die  skrywers  se  vanne  gerang- 
skik.  Let  op  die  volgende  voorbeelde: 

Cloete,  T.T.  1963.  Op  die  woord  af:  Opstelle  oor 
die  poësie  van  N.P.  van  Wyk  Louw.  Johan- 
nesburg,  Port  Elizabeth,  Kaapstad  en 
Bloemfontein:  Nasionale  Boekhandel. 
Coetser,  Johannes  Lodewikus.  1990.  Die 
geleentheidsdrama  by  N.R  van  Wyk 
Louw.  Pretoria:  Universiteit  van  Suid- 
Afrika.  (Ongepubliseerde  D.Litt.  et  Phil.- 
proefskrif.) 

Faverey,  Hans.  1993.  Verzamelde  gedichten. 

Amsterdam:  De  Bezige  Bij. 

Marais,  Johann  Lodewyk.  2005.  Die  belang- 
rikste  Afrikaanse  gedig(te).  [A].  Beskik- 
baar  by:  http://www.oulitnet.co.za/poesie/ 
default.asp  <Geraadpleeg  op  29  Junie 
2007>. 

Van  Coller,  H.R  & Odendaal,  B.J.  1999. 

George  Weideman  (1947-).  In:  Van  Coller, 
H.R  (red.).  Perspektief  en  profiel:  'n 
Afrikaanse  literatuurgeskiedenis:  Deel  2. 
Pretoria:  J.L.  van  Schaik,  pp.  764-785. 
Viljoen,  Louise.  2003.  Die  digter  as  reisiger: 
Twee  gedigsiklusse  van  Leipoldt  en  Krog. 
Stilet  15(1),  Maart,  pp.  80-100. 

Menings  wat  in  bydraes  uitgespreek  word, 
word  nie  noodwendig  deur  die  redaksie  onder- 
skryf  nie. 


Instructions  to  authors 


Ensovoort  gladly  receives  articles,  book  reviews 
and  creative  writing  (poems,  prose  and  short 
dramas)  for  publication.  The  focus  is  on  Afri- 
kaans  and  Dutch  literature  in  the  context  of 
both  the  broad  South  African  and  the  Dutch 
and  Flemish  communities.  Contributions  of 
sociocultural,  environmental  literature,  ap- 
plied  linguistics  or  comparative  nature  are  also 
applicable. 

Although  contributions  in  Afrikaans,  EXitch 
and  English  will  receive  priority,  contributions 
in  the  other  South  African  languages  will  also 
be  welcome. 

Ensovoort  hasbeen  accredited  for  subsidisa- 
tion  to  universities  and  research  outputs.  Page 
money  will  be  recovered  from  the  institutions 
of  writers  whose  contributions  qualify  for  sub- 
sidisation. 

Contributions  must  not  exceed  6 000  words 
and  must  be  sent  electronically  to  the  editor- 
ship,  typed  in  one  and  a halve  spacing  in  Times 
New  Roman  12-point  type-font  with  indent- 
ed  paragraphs  but  without  a blank  line  be- 
tween  paragraphs.  The  text  of  articles  must  also 
be  preceded  by  a translation  of  the  title  and  a 
summary  of  between  150  and  200  words  in 
English. 

Contributions  will  be  reviewed  by  at  least 
two  referees.  Although  the  referees'  reports 
can  be  sent  to  contributors,  the  identity  of  the 
referees  will  be  kept  confidential. 

All  contributions  must  be  ready  for  publi- 
cation  when  submitted  and  thoroughly  lin- 
guistically  edited.  High  standards  with  regard 
to  quality  and  linguistic  care  are  expected.  The 
editors  reserve  the  right  to  edit  contributions 
after  they  have  been  refereed  (if  necessary),  but 
the  individuality  of  authors  will  be  respected. 

The  Harvard  method  must  be  used  for  re- 
ferences.  References  in  the  text  must  consist 
of  the  name(s)  of  the  author(s),  year  of  publi- 


cation  and  page  number(s)  between  brackets, 
for  example  (Cloete,  1963:6). 

Numbers  of  endnotes  should  be  affixed  as 
superscripts  withoutbrackets  and  to  the  right 
of  any  punctuation  marks  in  the  text.  The  end- 
notes  appear  at  the  end  of  the  text  after  the 
bibliography. 

Full  details  of  the  literature  referred  to  in 
the  text  must  appear  at  the  end  of  the  article 
under  the  heading  "Bibliography".  The  litera- 
ture  must  be  arranged  alphabetically  accord- 
ing  to  the  authors'  surnames.  Please  note  the 
following  examples: 

Cloete,  T.T.  1963.  Op  die  woord  af:  Opstelle  oor 
die  poësie  van  N.P  van  Wyk  Louw.  Johan- 
nesburg,  Port  Elizabeth,  Kaapstad  and 
Bloemfontein:  Nasionale  Boekhandel. 
Coetser,  Johannes  Lodewikus.  1990.  Die 
geleentheidsdrama  by  N.E  van  Wyk 
Louw.  Pretoria:  Universiteit  van  Suid- 
Afrika.  (Unpublished  D.Litt.  et  Phil. 
thesis.) 

Faverey,  Hans.  1993.  Verzamelde  gedichten. 
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Marais,  Johann  Lodewyk.  2005.  Die  belang- 
rikste  Afrikaanse  gedig(te).  [A].  Available 
at:  http://www.oulitnet.co.za/poesie/ 
default.asp  <Accessed  on  29  June  2007>. 
Van  Coller,  H.P  & Odendaal,  B.J.  1999. 

George  Weideman  (1947-).  In:  Van  Coller, 
H.P  (red.).  Perspektief  en  profiel:  'n 
Afrikaanse  literatuurgeskiedenis:  Deel  2. 
Pretoria:  J.L.  van  Schaik,  pp.  764-785. 
Viljoen,  Louise.  2003.  Die  digter  as  reisiger: 
Twee  gedigsiklusse  van  Leipoldt  en  Krog. 
Stilet  15(1),  Maart,  pp.  80-100. 

Opinions  that  are  raised  in  contributions  are 
not  necessarily  endorced  by  the  editorship. 
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Academic  literacy  itself  is  no  longer 
the  domain  of  those  who  only  wish  to 
do  good  or  to  make  some  overt 
political  statement.  Rather,  it  is  a 
growing  field  involving  those  who 
intend  to  go  about  their  business  of 
facilitating  the  language  development 
of  students  in  as  theoretically  and 
socially  responsible  a way  as  possible 
...  teaching  on  its  own  will  never  be 
sufficient  to  give  an  adequate 
response  to  the  needs  of  the  students 
at  the  receiving  end  of  our  language 
instruction:  our  teaching  needs  to  be 
informed  by  research  of  the  highest 
quality  ... 
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